{"id":699,"date":"2020-05-02T07:01:46","date_gmt":"2020-05-02T07:01:46","guid":{"rendered":"https:\/\/www.peopleperhour.com\/engineering\/?p=699"},"modified":"2020-05-03T07:27:45","modified_gmt":"2020-05-03T07:27:45","slug":"testing-nginx-rewrite-performance","status":"publish","type":"post","link":"https:\/\/www.peopleperhour.com\/engineering\/2020\/05\/02\/testing-nginx-rewrite-performance\/","title":{"rendered":"Testing nginx rewrite performance"},"content":{"rendered":"\n

Our website is 12 years old and over time we have accured quite a few URL changes. Mainly due to terminlogy changes and SEO restructures. 819 to be exact. They are rewrite<\/code> rules in a nginx config file. This is a lot of regular expressions that nginx needs to execute every request. We were interested in the performance impact so we did a quick test.<\/p>\n\n\n\n

We created a minimum nginx config:<\/p>\n\n\n\n

$ cat \/home\/tom\/nginx_test\/nginx.conf\n\nevents {\n    use epoll;\n}\n\nerror_log   off;\n\nhttp {\n    access_log  off;\n\n    # This folder will have our rewrite rules in it\n    include \/etc\/nginx\/rewriterules\/*;\n\n    server {\n        listen 80;\n\n        # a self-contained endpoint without any dependencies on other services.\n        location \/ {\n          expires -1; # Disable client caching (Output Header: Cache-Control: no-cache)\n          default_type text\/plain;\n          return 200 ready;\n        }\n    }\n}<\/code><\/pre>\n\n\n\n
Lets check it works<\/p>\n\n\n\n
$ docker run --name loadtest -v \/home\/tom\/nginx_test\/nginx.conf:\/etc\/nginx\/nginx.conf:ro -d -p 49666:80 nginx;\n$ wget -qO- http:\/\/127.0.0.1:49666\/\nready<\/code><\/pre>\n\n\n\n
ok, we’re ready for our first test, which is the base-case, i.e. 0 redirects. We’ll use vegeta<\/a> to time the latency over 10,000 requests.<\/p>\n\n\n\n
$ date;echo 'GET http:\/\/127.0.0.1:49666\/' | vegeta attack -rate=1000 -duration=10s | vegeta report\nSat  2 May 07:46:32 BST 2020\nRequests      [total, rate, throughput]         10000, 1000.12, 1000.10\nDuration      [total, attack, wait]             9.999s, 9.999s, 170.026\u00b5s\nLatencies     [min, mean, 50, 90, 95, 99, max]  99.415\u00b5s, 154.843\u00b5s, 132.126\u00b5s, 197.848\u00b5s, 292.922\u00b5s, 498.15\u00b5s, 3.898ms\nBytes In      [total, mean]                     50000, 5.00\nBytes Out     [total, mean]                     0, 0.00\nSuccess       [ratio]                           100.00%\nStatus Codes  [code:count]                      200:10000<\/code><\/pre>\n\n\n\n
For test 2, we’ll add 819 rewrite rules to nginx and test again. I won’t show all our rewrite rules but here is a small extract to show an example of the kind of thing we’re talking about:<\/p>\n\n\n\n
$ cat \/home\/tom\/nginx_test\/rewriterules.conf\n...\nrewrite ^\/site\/aboutus$ https:\/\/$http_host\/about permanent;\nrewrite ^\/static\/how-it-works-buyer$ https:\/\/$http_host\/how-it-works permanent;\nrewrite ^\/static\/how-it-works-seller$ https:\/\/$http_host\/how-it-works permanent;\nrewrite ^\/site\/joinus$ https:\/\/careers.jobscore.com\/careers\/peopleperhourcom permanent;\n...<\/code><\/pre>\n\n\n\n
Lets run the same test, 10,000 requests again but this time I mount the rewriterules.conf<\/code> into the nginx docker container:<\/p>\n\n\n\n
$ docker stop loadtest;docker rm loadtest;docker run --name loadtest -v \/home\/tom\/nginx_test\/nginx.conf:\/etc\/nginx\/nginx.conf:ro -v  \/home\/tom\/nginx_test\/rewriterules.conf:\/etc\/nginx\/rewriterules.conf -d -p 49666:80 nginx;\n\n$ date;echo 'GET http:\/\/127.0.0.1:49666\/' | vegeta attack -rate=1000 -duration=10s | vegeta report\nSat  2 May 07:25:59 BST 2020\nRequests      [total, rate, throughput]         10000, 1000.11, 1000.09\nDuration      [total, attack, wait]             9.999s, 9.999s, 197.294\u00b5s\nLatencies     [min, mean, 50, 90, 95, 99, max]  144.79\u00b5s, 191.594\u00b5s, 172.16\u00b5s, 217.622\u00b5s, 255.267\u00b5s, 562.041\u00b5s, 3.485ms\nBytes In      [total, mean]                     50000, 5.00\nBytes Out     [total, mean]                     0, 0.00\nSuccess       [ratio]                           100.00%\nStatus Codes  [code:count]                      200:10000<\/code><\/pre>\n\n\n\n
The median latency has worsened from 132.126\u00b5s to 172.16\u00b5s. This is peanuts, but it shows the extra rewrites are having a impact to performance, however small.<\/p>\n\n\n\n
Lets finish up by trying a really large number of rewrites. I took my  rewriterules.conf<\/code> file, did select-all, then copy and paste, paste, paste*10 until I had 10,000 rules. I ran exactly the same command as the previous test:<\/p>\n\n\n\n
$ docker stop loadtest;docker rm loadtest;docker run --name loadtest -v \/home\/tom\/nginx_test\/nginx.conf:\/etc\/nginx\/nginx.conf:ro -v  \/home\/tom\/nginx_test\/rewriterules.conf:\/etc\/nginx\/rewriterules.conf -d -p 49666:80 nginx;\n\n$ date;echo 'GET http:\/\/127.0.0.1:49666\/' | vegeta attack -rate=1000 -duration=10s | vegeta report\nSat  2 May 07:29:11 BST 2020\nRequests      [total, rate, throughput]         10000, 1000.11, 1000.01\nDuration      [total, attack, wait]             10s, 9.999s, 934.258\u00b5s\nLatencies     [min, mean, 50, 90, 95, 99, max]  801.745\u00b5s, 8.855ms, 906.9\u00b5s, 28.047ms, 51.096ms, 74.329ms, 467.309ms\nBytes In      [total, mean]                     50000, 5.00\nBytes Out     [total, mean]                     0, 0.00\nSuccess       [ratio]                           100.00%\nStatus Codes  [code:count]                      200:10000  <\/code><\/pre>\n\n\n\n
The results<\/h2>\n\n\n\n
With no rewrites a page is served in 132.126\u00b5s. With 819 rewrites a page is served in 172.16\u00b5s. With 10,000 rewrites a page is served in 906.9\u00b5s.<\/p>\n\n\n\n
The conclusion<\/h2>\n\n\n\n
Ngix rewrites are super fast, use as many as you want because the performance impact is negligible. We’re talking nano<\/em>-seconds.<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
Our website is 12 years old and over time we have accured quite a few URL changes. Mainly due to terminlogy changes and SEO restructures. 819 to be exact. They are rewrite rules in a nginx config file. This is a lot of regular expressions…<\/p>\n","protected":false},"author":40,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[33],"tags":[14,24,34,46],"class_list":["post-699","post","type-post","status-publish","format-standard","hentry","category-nginx","tag-devops","tag-docker","tag-nginx","tag-performance"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2CA4w-bh","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts\/699","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/users\/40"}],"replies":[{"embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/comments?post=699"}],"version-history":[{"count":4,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts\/699\/revisions"}],"predecessor-version":[{"id":703,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/posts\/699\/revisions\/703"}],"wp:attachment":[{"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/media?parent=699"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/categories?post=699"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.peopleperhour.com\/engineering\/wp-json\/wp\/v2\/tags?post=699"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}