Support for plain nginx test backend

[margau]

Description

Providing an nginx configuration to act as basic test backend

Why it should be implemented

No further service needed, which allows a small-footprint test endpoint on "light" systems. For my side, I'm planning to deploy this on routers at different locations. Nginx is available, but I wanna to avoid setting up separate services trough the network

Implementation suggestions

Prototype with the provided nginx configuration is available under https://speedtest.margau.net (IPv6 only)

The only thing missing is the upload test, which i unfortunately not got working by now.

server {
  listen [::]:443 ssl http2 default_server ipv6only=off;

[...]
    location /speedtest/empty {
      client_max_body_size 256M;
      add_header Access-Control-Allow-Origin "*";
      add_header Access-Control-Allow-Methods "GET, POST";
      add_header Access-Control-Allow-Headers "Content-Encoding, Content-Type";
      add_header Cache-Control "no-store, no-cache, must-revalidate, max-age=0, s-maxage=0'";
      add_header Cache-Control "post-check=0, pre-check=0";
      add_header Pragma no-cache;
      return 200;
    }
    location /speedtest/dl {
      add_header Access-Control-Allow-Origin "*";
      add_header Access-Control-Allow-Methods "GET, POST";
      add_header Access-Control-Allow-Headers "Content-Encoding, Content-Type";
      alias /var/www/edge.as213204.net/html/test/100m.bin;
    }
    location /speedtest/ip {
      add_header Access-Control-Allow-Origin "*";
      add_header Access-Control-Allow-Methods "GET, POST";
      add_header Access-Control-Allow-Headers "Content-Encoding, Content-Type";
      types { } default_type "application/json; charset=utf-8";
      return 200 '
        {"processedString":"$remote_addr","rawIspInfo":""}
      ';
    }

[...]
}

If I manage to solve the upload test issue, I'll happily provide an PR.

[adolfintel]

Thank you. If you can get it working properly I'll add it to the wiki

[Draco-Zero]

I see that the problem with the download has already been solved, judging by the test https://speedtest.margau.net could you please provide an example configuration for nginx?

[brknkfr]

Thank you for your work already!

I tried this again to provide a nginx backend because I want to run librespeed on my 10Gbit/s router where PHP (with openssl generating the chunks to download) appeared to be a bottleneck.

So this is what I achieved and it seems to work quite well, even with the upload. It could be that meanwhile something changed with librespeed ...

Preparations

• Make sure that your librespeed domain name has an A and/or AAAA DNS record.
• Disable telemetry if not yet disabled (or still rely on PHP for telemetry).
• Create a 100 megabyte file (You can do that with the simple linux command dd if=/dev/urandom of=100MB.bin bs=1M count=100). The size should be the garbagePhp_chunkSize value specified in speedtest_worker.js.
• Specify custom values in speedtest_worker.js:

url_dl: "nginx/dl", // path to a large file or garbage.php, used for download test. must be relative to this js file
url_ul: "nginx/empty", // path to an empty file, used for upload test. must be relative to this js file
url_ping: "nginx/ping", // path to an empty file, used for ping test. must be relative to this js file
url_getIp: "nginx/ip", // path to getIP.php relative to this js file, or a similar thing that outputs the client's ip

NGINX configuration

The librespeed package is unzipped and prepared under /var/www/librespeed.domain.tld/speedtest and the previously prepared 100 megabyte file 100MB.bin too. General speedtest parameter for nginx are saved in /etc/nginx/snippets/speedtest-params.conf.

First we need some general nginx parameters in the file /etc/nginx/snippets/speedtest-params.conf:

add_header Access-Control-Allow-Origin "*";
add_header Access-Control-Allow-Methods "GET, POST";
add_header Access-Control-Allow-Headers "Content-Encoding, Content-Type";
add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0, s-maxage=0';
add_header Cache-Control 'post-check=0, pre-check=0';
add_header Last-Modified $date_gmt;
if_modified_since off;
expires off;
etag off;
gzip off;
brotli off;

Then we create the actual nginx virtual host configuration file:

server {
	listen 80;
	listen [::]:80;

	server_name librespeed.domain.tld;
	
	root /var/www/librespeed.domain.tld/speedtest;

	index index.html;

	client_max_body_size 21M;

	location = /nginx/ip {
		include /etc/nginx/snippets/speedtest-params.conf;
		default_type "application/json; charset=utf-8";
		return 200 '{"processedString":"$remote_addr","rawIspInfo":""}';
	}

	location = /nginx/ping {
		include /etc/nginx/snippets/speedtest-params.conf;
		add_header Connection keep-alive;
		return 200;
	}

	location = /nginx/dl {
		include /etc/nginx/snippets/speedtest-params.conf;
		add_header Content-Description 'File Transfer';
		add_header Content-Type 'application/octet-stream';
		add_header Content-Disposition 'attachment; filename=100MB.bin';
		add_header Content-Transfer-Encoding binary;
		sendfile on;
		alias /var/www/librespeed.domain.tld/speedtest/100MB.bin;
	}

	location = /nginx/empty {
		include /etc/nginx/snippets/speedtest-params.conf;
		return 200;
	}

	location / {
		   	try_files $uri $uri/ =404;
	}

	access_log /var/log/nginx/librespeed.domain.tld.access.log;
	error_log /var/log/nginx/librespeed.domain.tld.error.log;
}

server {
	# LibreSpeed shouldn't use HTTP/3 or HTTP/2 for more accurate traffic measuring
	http2 off;
	http3 off;
	listen 443 ssl;
	listen [::]:443 ssl;

	add_header Alt-Svc 'http/1.1=":$server_port"; ma=86400';

	server_name librespeed.domain.tld;

	add_header Strict-Transport-Security "max-age=15768000; includeSubDomains" always;
	
	root /var/www/librespeed.domain.tld/speedtest;

	index index.html;

	client_max_body_size 21M;

	location = /nginx/ip {
		include /etc/nginx/snippets/speedtest-params.conf;
		default_type "application/json; charset=utf-8";
		return 200 '{"processedString":"$remote_addr","rawIspInfo":""}';
	}

	location = /nginx/ping {
	        include /etc/nginx/snippets/speedtest-params.conf;
		add_header Connection keep-alive;
		return 200;
	}

	location = /nginx/dl {
		include /etc/nginx/snippets/speedtest-params.conf;
		add_header Content-Description 'File Transfer';
		add_header Content-Type 'application/octet-stream';
		add_header Content-Disposition 'attachment; filename=100MB.bin';
		add_header Content-Transfer-Encoding binary;
		sendfile on;
		alias /var/www/librespeed.domain.tld/speedtest/100MB.bin;
	}

	location = /nginx/empty {
		include /etc/nginx/snippets/speedtest-params.conf;
		return 200;
	}

	# Optimized ssl settings for fast speed
	ssl_session_timeout 1d;
	ssl_session_cache shared:ssl-nginx-librespeed:1m;  # about 4000 sessions
	ssl_protocols TLSv1.3;
	ssl_ecdh_curve X25519:prime256v1:secp384r1;
	ssl_prefer_server_ciphers off;

	ssl_certificate /etc/letsencrypt/live/domain.tld/fullchain.pem; # managed by Certbot
	ssl_certificate_key /etc/letsencrypt/live/domain.tld/privkey.pem; # managed by Certbot

	access_log /var/log/nginx/librespeed.domain.tld.access.log;
	error_log /var/log/nginx/librespeed.domain.tld.error.log;
}

This seems to work quite well with much lesser resources, although with this configuration you have to leave out telemetry and the ISP information. Maybe this could be added to the wiki? @sstidl (Sorry for pinging you.)

(In my case I still use PHP, but not for generating the chunk files, only for backend/getIP.php and to store telemetry which is still very lightweight.)

[sstidl]

@brknkfr which wiki page do you mean?

[brknkfr]

@sstidl, I just realized, there is no wiki. Maybe it's enough to just point to my comment with the nginx configuration somewhere in https://github.com/librespeed/speedtest/blob/master/doc.md

[sstidl]

Thanks for the example. I added a link to the doc.md

[bctrainers]

I went ahead and implemented the above configuration on a local server. After a small amount of testing on a virtualized machine with LibreSpeed, serving a pre-generated file at 100MB causes a fair amount of IRQ Queuing (ksoftirqd) on the disk. Setting it to smaller file sizes doesn't make much of a difference - seems to actually make it worse. Using caching techniques, such as to store the file in RAM, or using fstab 'things' (e.g., tmpfs /var/cache/nginx tmpfs rw,size=500M,noexec,nosuid,nodev 0 0), doesn't make much difference either - which seems bit awkward.

This issue becomes extremely evident when more than one session is active performing a speedtest. While this method does completely cut down the CPU-pegging issue that was observed with PHP and Nginx, this method will potentially cause issues with Disk I/O.