PHP at Etsy

• Linux, Apache, MySQL, PHP
• Memcache, Gearman, StatsD, Vitess, Redis, Kafka, Varnish
• GCP, Terraform

API First!

Framework?

We built our own over the years

(bad idea, don't do that)

Request Routing

https://etsy.com/awesome/123

.htaccess Apache rewrite rule

RewriteRule ^awesome/(\d+)$   /awesome.php?id=$1 [L,NC,QSA]

awesome.php

require 'bootstrap.php';
$request  = HTTP_Request::getInstance();
$response = HTTP_Response::getInstance();
$controller = new Awesome_Controller();
$controller->doCoolThings($request, $response);

Code/Awesome/Controller.php

class Awesome_Controller extends Controller_Base {
  public function doCoolThings($request, $response) {
    $id = $request->getGet('id', 0);
    if (!$id) {
      $response->redirect_error(Constants::ERROR_NOT_FOUND);
      return;
    }
    $thing = EtsyORM::getFinder('Thing')->findById($id);
    $stuff = Api::endpoint('AwesomeStuff', [$thing->id, 'max'=>10]);
    $this->renderViewTree(New Awesome_View($thing, $stuff));
  }
}

Awesome_View

class Awesome_View implements Neu_View {
    const TEMPLATE = "/templates/awesome/main.mustache";
    use Neu_Traits_DefaultView;

    public function __construct(AwesomeThing $thing, array $stuff) {
        $this->thing = $thing;
        $this->stuff = $stuff;
    }
    public function getCssFiles(): array {
        return [ '/awesome/main.scss' ];
    }
    public function getTemplateData(): array {
        return [ 'thing_id' => $this->thing->id,
                 'thing_name' => $this->thing->name,
                 'stuff' => $this->stuff ];
    }
}

templates/awesome/main.mustache

<div>
    <p>{{thing_name}} ({{thing_id}})</p>
    <ul>
    {{#stuff}}
        <li>{{id}} {{description}}</li>
    {{/stuff}}
    </ul>
</div>

Static Analysis

Install with composer

$ composer require --dev phan/phan

Create .phan/config.php

return [
    'target_php_version' => '8.2',
    'directory_list' => [ 'src/' ],
    "exclude_analysis_directory_list" => [ 'vendor/' ],
];

$ ./vendor/bin/phan

Phan in Browser

Dependency Graph Plugin

Daemon mode

$ phan --daemonize-tcp-port default &
[1] 28610
Listening for Phan analysis requests at tcp://127.0.0.1:4846
Awaiting analysis requests for directory '/home/rasmus/phan_demo'

$ vi src/script.php

$ phan_client -l src/script.php
Phan error: TypeError: PhanTypeMismatchArgument: Argument 1 (union) is array{0:1} but \C::fn() takes int|string defined at src/script.php:8 in src/script.php on line 14
Phan error: TypeError: PhanTypeMismatchArgument: Argument 3 (shaped) is array{max:10} but \C::fn() takes array{mode:string,max:int} defined at src/script.php:8 in src/script.php on line 16

vim integration

Let's deploy it!

Atomic

No performance hit

• No restarts
• No LB removal
• No thundering herd
• Cache reuse

Must be able to serve two versions of the site concurrently!

Requests that begin on DocumentRoot A must finish on A

Set the DocumentRoot to symlink target!

Easy with nginx

fastcgi_param DOCUMENT_ROOT $realpath_root

Apache

Avoid hardcoding full paths

Watch your include_path setting

incpath extension can resolve your include_path for you

Version all static assets

DB Schema changes need special care

Version Support

PHP 8.0

Named Arguments

htmlspecialchars($string, double_encode: false);

Constructor Property Promotion

class User {
    function __construct(public string $name, private string $pwd = "") { }
}

Nullsafe Operator with short-circuiting

$country = $session?->user?->getAddress(geoip())?->country;

Match Expression, Union Types

function days_in_month(string|int $month, int $year): int {
  $leap = $year % 400;
    $leap = !$leap|!($leap%4)&!!($leap%100); // 👀

  return match(is_string($month) ? strtolower(substr($month, 0, 3)) : $month) {
    'apr', 4, 'jun', 6, 'sep', 9, 'nov', 11  => 30,
    'jan', 1, 'mar', 3, 'may', 5, 'jul', 7, 'aug', 8, 'oct', 10, 'dec', 12 => 31,
    'feb', 2 => $leap ? 29 : 28,
    default => throw new InvalidArgumentException("Invalid month"),
  };
}

weakMap

- Map objects to arbitrary values without preventing GC

Attributes

- Structured metadata

function login(int $user_id, #[\SensitiveParameter] string $pwd) { }

PHP 8.1

Readonly properties

class Test {
  public readonly string $prop;

  public function __construct(string $prop) {
    $this->prop = $prop; // Initialized once in same scope
  }
}

$test = new Test("foobar");
var_dump($test->prop);
$test->prop = "foobar"; // Error

Enums

enum Suit {
  case Hearts;
  case Diamonds;
  case Clubs;
  case Spades;
}

function pick_a_card(Suit $suit) { ... }
pick_a_card(Suit::Clubs); // ok
pick_a_card('Spades');    // error

Fibers

Full-stack interruptable functions

use React\EventLoop\LoopInterface;
use React\Promise\PromiseInterface;

function await(PromiseInterface $promise, LoopInterface $loop): mixed {
  $fiber = Fiber::this();
  $promise->done(
    fn(mixed $value) => $loop->futureTick(fn() => $fiber->resume($value)),
    fn(Throwable $reason) => $loop->futureTick(fn() => $fiber->throw($reason))
  );

  return Fiber::suspend();
}

Change: Static Variable Inheritance

Inherited method shares parent's static vars

class A {
  public static function counter() {
    static $i = 0;
    return ++$i;
  }
}
class B extends A {}

echo A::counter();
echo A::counter();
echo B::counter();
echo B::counter();
// PHP 8.0 outputs 1212
// PHP 8.1 outputs 1234

No Return type

function redirect(string $uri): never {
  header('Location: ' . $uri);
  exit();
}

redirect('/index.html');
echo "this will never be executed!";

final for class constants

class Foo {
    final public const X = "foo";
}

class Bar extends Foo {
    public const X = "bar";
}

// Fatal error: Bar::X cannot override final constant Foo::X

new expressions can be used in initializers

class Test {
  public function __construct(private Logger $logger = new NullLogger) {}
}

// instead of

class Test {
  private Logger $logger;

  public function __construct(?Logger $logger = null) {
        $this->logger = $logger ?? new NullLogger;
    }
}

First-class callables

$fn = strlen(...);
$fn = $this->method(...)
$fn = Foo::method(...);

// instead of

$fn = Closure::fromCallable('strlen');
$fn = Closure::fromCallable([$this, 'method']);
$fn = Closure::fromCallable([Foo::class, 'method']);

Intersection types

class A {
  private Traversable&Countable $countableIterator;

  public function setIterator(Traversable&Countable $countableIterator): void {
    $this->countableIterator = $countableIterator;
  }

  public function getIterator(): Traversable&Countable {
    return $this->countableIterator;
  }
}

• Inheritance cache (avoid relinking classes)
• JIT improvements and add support for ARM64
• Optimize class name resolution

PHP 8.2

Readonly Classes

readonly class Test {
  public function __construct(public string $prop) { }
}
$test = new Test("Hi");
var_dump($test->prop);  // Hi
$test->prop = "foobar"; // Cannot modify readonly property Test::$prop

Prevents dynamic properties

Can't be used with untyped or static properties

Disjunctive Normal Form (DNF) Types

class A { }
class B extends A { };
class Foo {
    public function bar((A&B)|null $entity): A&B {
        if (!$entity) $entity = new B;
        return $entity;
    }
}
$c = new Foo;
$c->bar(new B);

8.0 Union Types X|Y

8.1 Intersection Types X&Y

8.2 DNF Types (X&A)|(Y&B)

Standalone null, false, and true types

class Falsy {
    public function alwaysFalse(): false { /* ... */ }

    public function alwaysTrue():  true  { /* ... */ }

    public function alwaysNull():  null  { /* ... */ }
}

New "Random" extension

$rng = $is_production
    ? new Random\Engine\Secure()
    : new Random\Engine\PCG64(1234);

$randomizer = new Random\Randomizer($rng);
$randomizer->shuffleString('Testing');

Provides multiple RNG engines as opposed to just using Mersenne Twister

Constants in Traits

trait T {
    public const CONSTANT = 1;
    public function bar(): int {
        return self::CONSTANT;
    }
}

class C {
    use T;
}

var_dump(C::CONSTANT); // 1

Sensitive Parameters

class User {
    function __construct(string $id, #[\SensitiveParameter] string $pwd) {
        throw new \Exception("Error");
    }
}
$u = new User('rasmus', 'very-secret');

Fatal error: Uncaught Exception
Stack trace:
#0 /home/rasmus/c(7): User->__construct('rasmus', Object(SensitiveParameterValue))
#1 {main}		

Deprecate dynamic properties

class User {
    public $name;
}

#[\AllowDynamicProperties]
class User2 { }

$user = new User();
$user->last_name = 'Doe'; // Deprecated notice

$user = new User2();
$user->last_name = 'Doe'; // ok

$user = new stdClass();
$user->last_name = 'Doe'; // ok

¡Aguas!

• Deprecated dynamic properties!!
• use Phan and #[\AllowDynamicProperties]
• Deprecated ${var} string interpolation
• use {$var}
• Deprecated utf8_encode() and utf8_decode()
• decode: use mb_convert_encoding($latin1, 'UTF-8', 'ISO-8859-1')
• encode: use mb_convert_encoding($utf8, 'ISO-8859-1', 'UTF8')
• Functions strtolower() and strtoupper() are no longer locale-sensitive
• Use mb_strtoupper()/mb_strtolower() if you need locale-aware conversion

Foreign Function Interface

Call a libc function

$ffi = FFI::cdef("int sched_getcpu(void);");
echo "Running on cpu " . $ffi->sched_getcpu();

Loading and calling library functions

$ffi = FFI::load("php_gifenc.h");

#define FFI_SCOPE "gifenc"
#define FFI_LIB "libgifenc.so"

typedef struct ge_GIF {
    uint16_t w, h;
    int depth;
    int fd;
    int offset;
    int nframes;
    uint8_t *frame, *back;
    uint32_t partial;
    uint8_t buffer[0xFF];
} ge_GIF;

ge_GIF *ge_new_gif(
    const char *fname, uint16_t width, uint16_t height,
    uint8_t *palette, int depth, int loop);
void ge_add_frame(ge_GIF *gif, uint16_t delay);
void ge_close_gif(ge_GIF* gif);

$ffi = FFI::load("php_gifenc.h");

$w = 240; $h = 180;
$cols = $ffi->new("uint8_t[12]");
/* 4 colours: 000000, FF0000, 00FF00, 0000FF */
$cols[3] = 0xFF; $cols[7] = 0xFF; $cols[11] = 0xFF;

$gif = $ffi->ge_new_gif("test.gif", $w, $h, $cols, 2, 0);
for($i = 0; $i < 16; $i++) {
    for ($j = 0; $j < $w*$h; $j++) {
        $gif->frame[$j] = ($i*6 + $j) / 12 % 8;
    }
    $ffi->ge_add_frame($gif, 5);
}
$ffi->ge_close_gif($gif);

Preloading FFI

/etc/php8/php-fpm-fcgi.ini:

ffi.enable=preload
ffi.preload=/var/www/html/FFI/*.h
opcache.preload=/var/www/html/preload.php
opcache.preload_user=www-data

preload.php:

foreach(glob("/var/www/html/FFI/*.php") as $file) {
    include $file;
}

cpp -P -C -D"__attribute__(ARGS)=" /usr/include/cpuinfo.h > cpuinfo-ffi.h

Edit cpuinfo-ffi.h and delete any inline code

Create a wrapper class

class CPUInfo {
    private static ?FFI $cpu = null;

    static function init() {
        if (self::$cpu) return;
        self::$cpu = FFI::scope("CPUINFO");
        self::$cpu->cpuinfo_initialize();
    }

    static function name() {
        self::init();
        $CData = self::$cpu->cpuinfo_get_package(0);
        return FFI::string($CData[0]->name);
    }

    static function __callStatic(string $function, array $args) {
        self::init();
        return self::$cpu->{'cpuinfo_'.$function}(...$args);
    }
}

Mapping a C struct

class CPUFreq {
    private static ?FFI $cpu = null;

    static function init() {
        if (self::$cpu) return;
        self::$cpu = \FFI::scope("CPUFREQ");
    }

    static function __callStatic(string $function, array $args) {
        self::init();
        $ret = self::$cpu->{'cpufreq_'.$function}(...$args);
        if ($ret instanceof FFI\CData) {
            switch (FFI::typeof($ret)->getName()) {
                case 'char*': $ret = FFI::string($ret); break;
                case 'struct cpufreq_policy*':
                    $ret = [ 'min' => $ret->min,
                             'max' => $ret->max,
                             'gov' => FFI::string($ret->governor) ];
                    break;
            }
        }
        return $ret;
    }
}

Calling it

$cpu_name = CPUInfo::name();
$threads = CPUInfo::get_processors_count();
$cores = CPUInfo::get_cores_count();
echo "$cores-core $cpu_name $threads threads\n";

$cpu = 0;
while(CPUFreq::cpu_exists($cpu) == 0) {
    $driver = CPUFreq::get_driver($cpu);
    $policy = CPUFreq::get_policy($cpu);
    $governor = $policy['gov'];
    $min = sprintf("%.2f", $policy['min']/1000);
    $max = sprintf("%.2f", $policy['max']/1000);
    echo "CPU " . sprintf("%02d", $cpu) .
         " $driver $governor ($min MHz - $max MHz): " .
         sprintf("%.2f", CPUFreq::get_freq_kernel($cpu)/1000) . " MHz\n";
    $cpu++;
}

16-core AMD Ryzen 9 3950X 32 threads
CPU 00 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 1555.64 MHz
CPU 01 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 3500.24 MHz
CPU 02 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 2333.11 MHz
CPU 03 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 1555.63 MHz
CPU 04 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 623.08 MHz
CPU 05 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 06 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 07 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 08 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 634.24 MHz
CPU 09 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 634.24 MHz
CPU 10 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 592.12 MHz
CPU 11 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 592.12 MHz
CPU 12 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 2314.04 MHz
CPU 13 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 14 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 15 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 921.44 MHz
CPU 16 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 1613.96 MHz
CPU 17 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 1960.96 MHz
CPU 18 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 2330.25 MHz
CPU 19 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 1555.87 MHz
CPU 20 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 1389.00 MHz
CPU 21 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 557.21 MHz
CPU 22 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 23 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 24 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 622.76 MHz
CPU 25 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 26 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 592.12 MHz
CPU 27 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 550.00 MHz
CPU 28 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 854.47 MHz
CPU 29 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 799.01 MHz
CPU 30 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 642.85 MHz
CPU 31 amd-pstate ondemand (550.00 MHz - 4762.00 MHz): 844.84 MHz

35+ years!

• Bell Northern Research - Toronto
• Northern Telecom - Toronto
• Digital Media Networks - Toronto
• NovAtel - Calgary
• Nutec Informática - Porto Alegre, Brazil
• University of Toronto IT - Toronto
• Bell Global Solutions - Toronto
• IBM - Raleigh, NC
• Linuxcare - San Francisco
• Yahoo! - Sunnyvale
• WePay - Palo Alto
• Etsy

Create more value than you capture. -Tim O'Reilly

Work on things that matter (to you)

¡Gracias!

Interested in joining our team?

Please email your CV/resume to

mexico-hiring@etsy.com