If you tell me that you can program a FTP server in PHP 5 years ago, I would probably say that it is impossible. The reason why? FTP requires parallel connections and you can not run anything parallel in PHP. However I played with ReactPHP and found out that you CAN make a FTP server in PHP.
Parallel execution
So PHP is made for executing code one statement at the same time. It will only execute the next statement if the previous function is completed. For example if I would download 3 files in PHP they will be executes one by one and thus very slow:
file_put_contents('output/0.png', file_get_contents('http://example.com/image/0.png'));
file_put_contents('output/1.png', file_get_contents('http://example.com/image/1.png'));
file_put_contents('output/2.png', file_get_contents('http://example.com/image/2.png'));
It would download the image, wait for the download to succeed and then stores the file contents in a local file where it will also wait for it to be finished. It would be really faster to run 3 PHP processes with a script like this:
$file = $argv[1];
file_put_contents('output/' . $file . '.png', file_get_contents('http://example.com/image/' . $file . '.png'));
I could execute this parallel by starting 3 PHP processes at the same time:
php myscript.php 0 &
php myscript.php 1 &
php myscript.php 2 &
For long I thought this was the only way to run things in parallel: just start a new PHP process. Of course starting a PHP process comes with a large and slow starting process.
Threads
Besides starting a new process, you could run code in threads. A thread shares the memory with other threads in the same process and memory can also be used to communicate between threads. If we want to use threads in PHP you end up with the pthreads php extension, which have some very low-level functions. The biggest challenge with threads is when multiple threads change the same objects. In those cases you could run into racing conditions and have to lock your critical actions. If you use multiple locks you also have to be careful that you do not create a deadlock.
Event loops
If you have worked with javascript or programmed in C(++) a graphical user interface you might have run into a 3rd option to run things in parallel. An event loop mimics it is running multiple things at the same time, but in reality it still runs one line of code one by one. In fact: the event loop is so convincing that many developers actually think that javascript runs code in parallel. An event loop is much easier to grasp. A downside is that if something is slow in execution it still has to wait and the illusion of parallel processing disappears.
In Node if you download a url asynchronously what really happens it that it will execute the rest of the code and download your file in a background thread. When the download is ready it will send a notification the file was downloaded.
In the background Node.js uses a special open source library called libuv that handles any IO in a background process. In theory you could make a php extension from it or use FFI to connect libuv with PHP, but this is not needed at all!
In PHP we could also implement an event loop with a library like ReactPHP that requires no php extensions at all. The only difference with PHP is that since it's not running an event loop by itself, it needs to be called manually in PHP:
Loop::get()->run();Implementing FTP
FTP is a very ancient protocol. You just need to run a server that handles HTTP (the protocol). You give small instructions over HTTP and the FTP server will response with a 3-digit response followed by a text of the response. This is called the control connection. To download or upload a file or directory contents, you need to run command in the control connection to start up a connection for transfering data.
use React\EventLoop\Loop;
use React\Socket\ConnectionInterface;
use React\Socket\SocketServer;
$loop = Loop::get();
$port = 21;
$server = new SocketServer('0.0.0.0:' . $port);
$server->on('connection', function (ConnectionInterface $conn) {
$this->handleConnection($conn);
});
$loop->run();FTP Commands are always written in this structure:
CMD <arg> <arg>. There is a RFC for FTP with the minimal set of commands, called RFC 959. It's very simple to parse a FTP command:
private function handleConnection(ConnectionInterface $conn)
{
$conn->write("220 Apie FTP Server Ready\r\n");
$context = new ApieContext([ConnectionInterface::class => $conn]);
$conn->on('data', function ($data) use ($conn, $output, &$context) {
$command = trim($data);
[$cmd, $arg] = array_pad(explode(' ', $command, 2), 2, null);
$cmd = strtoupper($cmd);
$context = $this->runner->run($context, $cmd, $arg ?? '');
});
}So the parsing is done with explode() on a single space and only do it once, with
[$cmd, $arg] = array_pad(explode(' ', $command, 2), 2, null);. The runner code works a bit like this:
public function __construct(
private readonly CommandHashmap $commands
) {
}
public static function create(SiteCommandInterface... $siteCommands): self
{
return new self(
new CommandHashmap([
'CDUP' => new CdupCommand(),
'CWD' => new CwdCommand(),
'EPRT' => new EprtCommand(),
'EPSV' => new EpsvCommand(),
'LIST' => new ListCommand(),
'NLST' => new NlstCommand(),
'PASS' => new PassCommand(),
'PASV' => new PasvCommand(),
'PBSZ' => new PbszCommand(),
'PORT' => new PortCommand(),
'PROT' => new ProtCommand(),
'PWD' => new PwdCommand(),
'QUIT' => new QuitCommand(),
'RETR' => new RetrCommand(),
'SITE' => SiteCommand::create(...$siteCommands),
'SYST' => new SystCommand(),
'TYPE' => new TypeCommand(),
'USER' => new UserCommand(),
])
);
}
public function run(ApieContext $apieContext, string $command, string $arguments = ''): ApieContext
{
if ($command === 'FEAT') {
$apieContext->getContext(ConnectionInterface::class)->write("211-Features:\r\n");
foreach ($this->commands as $commandName => $commandExecutable) {
if ($commandExecutable instanceof FtpFeatureCommand) {
$helpText = implode(' ', $commandExecutable->getFeatures()->toArray());
$apieContext->getContext(ConnectionInterface::class)->write("211-$commandName $helpText\r\n");
} else {
$apieContext->getContext(ConnectionInterface::class)->write("211-$commandName\r\n");
}
}
$apieContext->getContext(ConnectionInterface::class)->write("211 End\r\n");
return $apieContext;
}
if (!isset($this->commands[$command])) {
$apieContext->getContext(ConnectionInterface::class)->write("502 Command not implemented\r\n");
return $apieContext;
}
$commandExecutable = $this->commands[$command];
return $commandExecutable->run($apieContext, $arguments);
}
}
This code is not that hard to grasp. I added a static create method for a general FTP server runner, but you could also call the constructor for a complete custom FTP server. I added a hardcoded FEAT command which FTP Clients can use to ask the available commands they can run. The Context variable is to keep track of the current state of the connection, for example if I run the USER command I expect to setup an username to connect, but the command only starts doing something if the PASS command is used as well. This only works if the context object is immutable to make sure it is different per connection!
class UserCommand implements CommandInterface
{
public function run(ApieContext $apieContext, string $arg = ''): ApieContext
{
$conn = $apieContext->getContext(ConnectionInterface::class);
if ($arg) {
$conn->write("331 Username OK, need password\r\n");
return $apieContext
->withContext(FtpConstants::USERNAME, trim($arg));
} else {
$conn->write("530 Login incorrect.\r\n");
}
return $apieContext;
}
}
SITE is a special FTP command to run custom commands. Most of the time it does things for changing folder ownership or do an idle call.
class IdleCommand implements SiteCommandInterface
{
public function getName(): string
{
return 'IDLE';
}
public function getHelpText(): string
{
return 'Waits for a short moment before responding.';
}
public function run(ApieContext $apieContext, string $arg = ''): ApieContext
{
$conn = $apieContext->getContext(ConnectionInterface::class);
$conn->write("200 Idle for 1 usec\r\n");
usleep(1);
return $apieContext;
}
}
So to test my FTP server I made an integration test that runs a separate PHP process that runs a FTP server and use the PHP FTP functions to see if PHP can connect with them:
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\Attributes\RequiresPhpExtension;
use PHPUnit\Framework\TestCase;
use Symfony\Component\Process\PhpProcess;
class IntegrationTest extends TestCase
{
private PhpProcess $process;
protected function setUp(): void
{
$this->process = new PhpProcess(file_get_contents(__DIR__ . '/run-server.php'), __DIR__);
$this->process->start();
// Wait a moment for the server to start
sleep(1);
if (!$this->process->isRunning()) {
$this->markTestSkipped('Could not run FTP server for test: '. $this->process->getErrorOutput() . $this->process->getOutput());
}
}
protected function tearDown(): void
{
$this->process->stop(2);
}
#[RequiresPhpExtension('ftp')]
#[DataProvider('listFoldersProvider')]
public function testListFolders(bool $passive)
{
$ftp = ftp_connect('127.0.0.1', 2121, 10);
$this->assertNotFalse($ftp, 'Could not connect to FTP server: ' . $this->process->getErrorOutput() . $this->process->getOutput());
try {
// Try anonymous login (since no credentials are set up)
$login = @ftp_login($ftp, 'anonymous', '');
$this->assertTrue($login, 'Could not login to FTP server: ' . $this->process->getErrorOutput() . $this->process->getOutput());
ftp_pasv($ftp, $passive);
// List files in the root directory
$files = ftp_nlist($ftp, '/');
if ($passive && $files === false) {
$this->assertStringContainsString("Nobody connected with the server within the timeout period", $this->process->getErrorOutput());
} else {
$this->assertIsArray($files, 'ftp_nlist did not return an array: ' . $this->process->getErrorOutput() . $this->process->getOutput());
// Optionally, assert something about the files, e.g. not empty
$this->assertEquals(['default', 'other'], $files, '2 files found in root directory');
}
} finally {
ftp_close($ftp);
}
}
The test initially failed, because I really need to implement the commands PASV and PORT. A FTP Client will use these commands to indicate how to upload or download files. With PORT the server connects to the client on a specific IP address and port. With PASV the server starts listening to a port and the client will send/retrieve data on this port. Both are possible in ReactPHP very easily:
// PORT connection:
$connector = new Connector();
$connector->connect($ip . ':' . $port);
$connector->write(file_get_Contents(__DIR__ > '/file.txt'));
// PASSIVE connection:
$pasvServer = new SocketServer('0.0.0.0:' . $dataPort);
$pasvServer->on('connect', function (ConnectionInterface $connection) {
$connection->write(file_get_Contents(__DIR__ > '/file.txt'));
});
Testing with Filezilla!
I did manage to make the test succeed! And using ftp from the terminal also works! So I tried to connect with Filezilla (a FTP Client). And it fails! The reason: Filezilla uses the extended EPASV and EPRT over PASV and PORT to send an ipv6 address. This is from a different RFC: RFC 2428. To simplify a little bit I do not add ipv6 support and just only accept ipv4 addresses.
FTP over SSL
As you see from the screenshot of Filezilla, unencrypted FTP is insecure. Sending a password and username with FTP without SSL means everybody can read the password you used. So is it possible to add SSL support? FTP over SSL is more secure, but it does not support identification. The only thing FTP Clients can do is store the certificate themselves. But as we will see it is much easier to setup FTP over SSL because of this.
FTP over SSL is available in 2 flavours: Implicit SSL and Explicit SSL
FTP over implicit SSL
When you use this all communication is handled through SSL. ReactPHP has support for this. We just need to wrap our server with a SecureServer class:
$server = new SecureServer(
new SocketServer('0.0.0.0:' . $port),
null,
$this->createSslOptions()
)
This function needs to create an array with SSL options, for example where it can find the certificate. We can use a self-signed certificate here as FTP does not use it for identification. I checked the source code of ReactPHP to find an example how to generate a self-signed certificate for localhost in PHP.
/**
* @return array<string, mixed>
*/
private function createSslOptions(): array
{
$crtFile = sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'server.crt';
$keyFile = sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'server.key';
if (!file_exists($crtFile) || !file_exists($keyFile)) {
$dn = [
"commonName" => "localhost",
];
$privkey = openssl_pkey_new();
$cert = openssl_csr_new($dn, $privkey);
$cert = openssl_csr_sign($cert, null, $privkey, 3650);
openssl_x509_export($cert, $out);
file_put_contents($crtFile, $out);
openssl_pkey_export($privkey, $out);
file_put_contents($keyFile, $out);
}
return [
'local_cert' => $crtFile,
'local_pk' => $keyFile,
'allow_self_signed' => true,
'verify_peer' => false,
];
}
For downloads and uploads we need to implement RFC4217 which contains some FTP commands to setup a secure connection. I decided to enforce encryption if encryption is enabled. I also randomize the used port so it is less predictable which port number is being used for uploading/downloading data.
Testing FTP Over SSL is harder as the standard PHP functions do not work, including the ftp_ssl functions (those only work with FTP over explicit SSL). The only way for PHP to connect is with Curl. Curl is stricter than the PHP functions, so I needed to make a few small changes to the responses.
FTP Over Explicit SSL
I have not found a way to program FTP Over Explicit SSL. The reason why is that you connect with FTP unencrypted and make a command to switch to SSL. Codewise I wrap SecureServer or not, so I do not know how to enable/disable this part. I'm also struggling with making a passive connection over implicit SSL, so for now I have it on hold.
Conclusion
I'm surprised, but it really is possible to make a working FTP server with parallel downloads and uploads. I do have to say I'm not sure it is production ready yet to use PHP with ReactPHP as a FTP server. You also need to get very low level to make it work. Also FTP is so ancient that every FTP client needs to be tested individually as they use different FTP commands and respond differently to errors. It certainly was fun to play with it for a change.
Comments
Post a Comment