It's All About the Goto

• It's a big state machine starting with INITIAL

• States: INITIAL, ST_IN_SCRIPTING, ST_DOUBLE_QUOTES, ST_NOWDOC…

• Tokens can change the state:

&lt;INITIAL>*"&lt;?php"*([ \t]|{NEWLINE}) {
    HANDLE_NEWLINE(yytext[yyleng-1]);
    BEGIN(ST_IN_SCRIPTING);
    RETURN_TOKEN(*T_OPEN_TAG*);
}