Timeline

G-WAN v1.1.61/Linux: Development Release

• added a JSON parser / renderer made available for C scripts
  (a dedicated example will come for Christmas)
• added on-the-fly CSS/JS/HTML reduction (blanks, comments...)
• added on-the-fly CSS Link URI to Data URI conversion for icons
• removed the mlockall() call which makes the daemon mode fail
  (making "connection resets" happen unexpectedly, no clue why
  yet, but thanks a bunch to 'Scott' for signaling a problem)
• added idiot-proof '-r' support for Tamás TEVESZ, fan of FUD.

Before 1.1.61, C servlets run with 'gwan -r' crashed because the server is not running (the manual states that the -r switch runs stand-alone general-purpose C source code rather than G-WAN C servlets). With the new dedicated G-WAN warning, even Tamás TEVESZ, a team member of the slower-than-Apache-1.3 Caudium web server, will no longer be able to shoot himself in the foot.

In the same vein, a "Web Hosting Guru" puzzled a G-WAN user by stating that "[G-WAN] benchmarks are misleading, you can achieve similar performance by writing modules for Apache, nGinx, Lighttpd etc. or by using libevent/evhttp directly".

False: because Apache, nginx, Lighttpd, libev, etc. are all slower than G-WAN for static contents and C scripts are almost as fast as GCC compiled modules, no force on Earth will make those servers faster than G-WAN for dynamic contents.

Even for heavy 3D (CPU intrinsics) computations, G-WAN C scripts could just use '#pragma link' to use an OpenGL library (supporting specific CPU instructions) to beat any (incomparably more complex) Apache or nginx pre-compiled module.

And G-WAN C scripts do it in 7 lines of code (without any stop or configuration):

1
2
3
4
5
6
7

#include "gwan.h" // G-WAN definitions

int main(int argc, char *argv[])
{
   xbuf_cat(get_reply(argv), "Hello World");
   return 200; // return an HTTP status (200:'OK')
}

Compare that to the Apache and nginx 'hello world' modules which force you to:

• create/edit the module C source code, (just like the shorter G-WAN C script)
• + stop the (Apache or nginx) Web server,
• + compile the module (Apache) or re-compile the Web server (nginx),
• + edit the Web server configuration files for any newly created module,
• + restart the Web server to use the newly created module.

You have to do this all before you can start testing your code. If there's a bug, or a feature that you want to enhance, then you have to do it all again.

With a G-WAN C script, you press F5 in the browser and the newly edited script is executed on-the-fly - considerably increasing your productivity.

G-WAN v1.1.6/Linux: Development Release

• removed the set_reply() call, see below to use the new get_reply():

  1 2 3 4 5 6 7 8

  xbuf_t reply; get_reply(argv, &reply); // before xbuf_t *reply = get_reply(argv); // now xbuf_cat(&reply, "Hello World!"); // before xbuf_cat(reply, "Hello World!"); // now set_reply(argv, &reply); return 200; // before return 200; // now

• added the full URL (with parameters) in C script crash reports
• added the "Vary: Accept-Encoding" HTTP header for proxy servers
• used parameterized queries to prevent SQL injection in sqlite.c
• huge dynamic buffers (like loan 1,000 years) that were not freed
  because of a script timeout are now released.

Edit your servlets when they say: undefined symbol 'set_reply' to match the new get_reply() usage (see above) and remove all set_reply() calls.

In C scripts, set_reply() was sometimes missing before a return(), leading to nasty bugs. Using an xbuf_t pointer for get_reply() also makes it (much) easier for G-WAN to control the memory allocated by C scripts.

Thanks to Alex Gacovski for his feedback about the loan(999) issue. Because of floating point overflows, a 999-year loan loops forever (G-WAN detects this and timeouts), and that (rather rare condition) was potentially leaving unreleased buffers before v1.1.6.

G-WAN v1.1.5/Linux: Development Release

• added the Host Alias feature (map domain names to root host)
• added syscall and libc symbol lookups in C script crash reports
• added the C 'continuations' (request.c, getheaders.c, attack.c)
• kill the daemon when gwan can't listen or when script is buggy
• fixed an epoll issue with 'ghost' events raising the CPU usage
• removed close() time-outs from log files (all the clients do that)
• added ac email 'attachment' feature to the sendemail() function
  (I still have to test it and to document it).

The 'continuations' are really black-magic: they let you write procedural code in C scripts (G-WAN Servlets and Handlers) but work asynchronously behind the scene so G-WAN's threads are never blocking.

This is doing marvels with G-WAN sending requests to database servers, or other back-end servers (G-WAN used as a reverse-proxy to feed G-WAN or another application server/script engine).

And... it should work transparently with all the existing network libraries! (I only implemented the client-side since G-WAN does the server-side but I could add an asynchronous FILE SYSTEM support for all among you who are accessing remote file systems).

More will come soon about this new feature, with real-life examples.

G-WAN v1.1.4/Linux: Production Release

• added a 'grace' delay for 'interactive' (save as) downloads
• added average traffic and gwan memory usage in HTML reports
• added local time GMT+offset in the gwan.log file and HTML reports
• tuned dynamic buffers to release memory faster than in the past.

G-WAN v1.1.3/Linux: Development Release

• fixed the PDF manual fonts mess made by Adobe Acrobat 7.0
  (now Windows users can read the Linux PDF manual)
• added a message explaining why -k can fail (lack of privileges)
• check for existence of the group / user profile specified with -k
• added DOWNLOAD_SPEED env. variable to tweak the DoS shield.

Thanks to Alex Gacovski for the testing and user-friendly warnings suggestions and to Jym Cheong for the DoS (constructive) criticism!

G-WAN v1.1.2/Linux: Development Release

• added a warning if -d is used without 'sudo' or the 'root' account
• a readlink() failure prevented -k from finding pid files, patched it
• restored unicode support in query strings (feedback is welcome).

Thanks to Alex Gacovski for the two first issues and to Jym Cheong for unicode!

G-WAN v1.1.1/Linux: Development Release

• added a gwan/trace file to store the start/stop status of the process
• added the parent and child uptimes for reports made in daemon mode
• added times a (daemon) parent forked a child in reports and gwan.log
• fixed a decimals error for the s_snprintf()/xbuf_xcat() "%llk" formater
• fixed the inverted CC_BYTES_INDAY/OUTDAY perfomance counters.

Now G-WAN/Linux is keeping our most hysterical visitors at bay, I will have time to work on the SSL/TLS and C continuation features and deliver the SCGI and Reverse-Proxy example Handlers. A real-life 'forum' C application will follow to illustrate how to make it all work together.

And if someone knows how to make statvfs64() work on Linux v#1 SMP Thu Sep 16 15:56:38 UTC 2010 built 2.6.26-2-amd64 (ia32) then please let me know.

G-WAN v1.1.0/Linux: Development Release

• added sendemail() an SMTP client supporting 'auth login' for C script
• added SHA1/SHA2 hashing and AES encryption functions for C script
• added Linux 'Cpus_allowed' confinement messages in logs and reports
• added list of network interfaces in gwan.log, help and error messages
• added CPU load (1/5/15min history) and system uptime in logs/reports
• added a "/csp/" alias of the "/csp?" prefix for RESTFUL Web services
• enhanced the thread scheduling policy to shorten latency a bit further
• enabled timeout Denial of Service attack logging in gwan error.log files
• HTTP Keep-Alives no longer shortened to 2 seconds (a v1.0.97 glitch)
• xbuf_frurl() now avoids infinite redirection loops from 'buggy' servers
• extended the Common Log Format (CLF) to the Combined Log Format
• applied stricter Unix permissions to the gwan/... files and folders
• fixed a servlet renaming glitch (thanks to Alex Gacovski's bug report)
• fixed the virtual hosts feature (thanks to Bob Stanton's bug report)
• fixed -d:group:user (thanks to Anton Tameev and Alex Gacovski reports)
• servlets can send JSON/XML/etc. bare contents (no HTTP headers or
  HTML body) by returning an invalid HTTP status code in the 1-99 range.

On Ubuntu 8.10 32-bit (Desktop), 'Cpus_allowed' (/proc/<pid>/status) restricts gwan to 2 Cores (unless you run 'sudo gwan'). G-WAN detects this now and reports it into the parent Terminal (if any), daily server report, and gwan.log file (letting you know why only one single physical Core may be involved: with hyper-threading, each physical Core counts for two logical Cores, and the Linux scheduler happens to jail a process into such a small environment even if, like me, you have 4 physical Cores begging for work).

I expected G-WAN v1.1 to provide SSL/TLS and continuations (the code is here, hence the extra bloat, but it is not used yet) allowing easy handler implementations like SCGI and Reverse-Proxy but the two last days kept me busy with nocturnal attacks on the new Linux server that hosts gwan.ch.

On October 11th, 16 of my domains were hijacked (like millions of other domains) by DNSmadeEasy. I have had to migrate my DNS and Web servers. Doing so has delayed the new G-WAN version, but G-WAN will also benefit from being used for production on a Linux 64-bit server. Good-bye Windows Server 2003, hello Linux.

G-WAN v1.0.97/Linux: Development Release

• restored adaptative time-outs (a glitch since v1.0.6 disabled them).

A lack of adaptative time-outs does not have consequences on performances (G-WAN time-outs can be disabled with the '-b' command-line switch) but it makes Web servers like Apache (300 seconds) or IIS (120 seconds 'after accept') vulnerable to denial of service attacks that open connections and send/receive data much too slowly (saturating your Web server's memory) or never send a complete HTTP request (saturating the system memory).

Apache and IIS neglect this deadly issue to the point where a single ADSL client can put down the so-called 'rock-solid' servers.

Slowloris is a breeze compared to Cyveillance robots which are more agressive and sneaky (but this is not visible in Web server logs so nobody complains).

G-WAN uses adaptative time-outs (rather than fixed-size time-outs) to defeat all the possible cases. This mechanism allowed G-WAN/Windows to survive millions of such (duly logged) attacks since July 2009 -and it is now available in G-WAN/Linux (just in case, like me, you want to stay online).

G-WAN v1.0.96/Linux: Development Release

• added script support for '#pragma link' (for .c, .obj, .a/.lib, .so/.dll files)
• added script support for '#pragma include' (for additional include paths)
• created a gwan/include folder to store all the files shared by servlets
• added servlet support for md5/crc32/adler32 and gzip/lzjb compressors
• added log_err() to let servlets dump text in their virtual host's error.log
• new examples: trace.c (tracing code) and sqlite.c (SQL database engine).

Many libraries are already installed in /usr/lib on Unix systems. Now, G-WAN lets you play with them by just using a couple of #pragmas (see the sqlite.c source code).

Look at the XAMPP or PHP projects -and let G-WAN use the same C libraries to deliver features like databases (gdbm, SQLite, mySQL, PostgreSQL), XML (Expat, libxml), RegEx, JSON or XML parsing, PNG/JPEG/MPEG, VoIP, mCrypt, mHash, FreeType (TrueType fonts), fPDF and Ming (Flash), SMTP/POP3/IMAP, OpenLDAP, CURL, zZIP, etc.

G-WAN v1.0.95/Linux (version unchanged, build date updated): Development Release

• get_env(argv, REMOTE_ADDR, 0); reported a wrong IP address if
  any of the a.b.c.d items was > 199; that was because of a typo (a
  missing comma) and this is now corrected.

Thanks to 'Jason' for the bug report!

G-WAN v1.0.95/Linux: Development Release

• gif_parse() (see the 'chart.c' example) parses in-memory GIF images
• gif_build() (see the 'fractal.c' example) now supports GIF comments
• added dr_chart() for bar/line/area/dot/pie/ring charts and sparklines
• added dr_text() to print text in bitmaps using bitmap/TrueType fonts
• added dr_line(), dr_circle(), dr_rect()/dr_recf() (handy to have them)
• added dr_gradient() for multi-gradient palettes based on RGB values
• added sw_rnd() a 2^158 PRNG, and hw_rnd() a fast Hardware RNG
• added get_env() USER_AGENT and FNT_ROOT 'enviroment' variables
• better documented G-WAN exported functions in the gwan.h header
• headers (gwan.h, xbuffer.h...) are now stored once: in the gwan folder
• get_env(argv, REMOTE_ADDR, 0); no longer gives a constant 0.0.0.0
• cacheadd() no longer fails when called successively for a same entry
• reduced even further the CPU load by replacing arithmetic routines
• the '-d' command-line switch now lets you specify a 'group' / 'user' to
  dump 'root' privileges when using the daemon mode.

Productivity: C scripts allowed to develop the charts library rapidly. It was nice to write code and press 'F5' in the Internet browser to see the result (no wait with compilation/linking and no need to use a tool like GIMP to manually open every new GIF file made on-the-fly). Once it worked, the code was then moved into gwan (where, good news, it did not execute any faster).

dr_chart() makes sparklines too (see the chart.c source code)

Coming next: LISP/SCHEME (true) 'continuations' and 'closures' - for ANSI C. Waving wait-states will speed-up client requests initiated by G-WAN (and C scripts) to back-end servers like database, email, proxy or other app. servers.

G-WAN v1.0.9/Linux: Development Release

• gif_build() (see the 'fractal.c' example) makes on-the-fly GIF images
• xbuf_xcat() with "%B" encodes a string in base64 ("%-B" to decode)
• improved source code line numbers lookup in C servlet crash reports
• fixed: POST HTTP requests were improperly treated as HEAD requests.

gif_parse() will complete gif_build() to let us print text in bitmaps with bitmap fonts (made from TrueType fonts). Thanks to 'Dopey' for the bug report!

G-WAN v1.0.8/Linux: Development Release

• maintenance scalability: the impact of web sites file updates is now null
• fixed issue: servlets updated with a bug can no longer raise a segfault.

Thanking 'SaltwaterC' for the very precise and relevant bug report!

G-WAN v1.0.7/Linux: Development Release

• made gwan work on virtualization layers (got memory allocation issues)
• the '-d' and '-k' command line switches now work in every possible case
• gwan no longer stops working after first servlet on a single-core system
• corrected the CPU brand string in reports and logs (a cosmetic change)
• xbuf_xcat(&buf, "%k", 1024); gives "1.00 KB" ("%llk" for 64-bit values).

Thank you for the feedback that made it possible to identify various issues!

G-WAN v1.0.6/Linux Development Release (you *must* read the updated manual):

• IP and domain-name based virtual hosts support (no configuration files)
• implemented awaited handlers, the maintenance script, and persistence
• added cacheadd() and cachedel() to let servlets store entries in cache
• added getms() and getus() to get current time in millisec and microsec
• added gzip+deflate on-the-fly compression (server replies >100 bytes)
• extended get_env() to access internal performance options/counters
• added a '-b' command-line switch to disable the Denial of Service shield
• added a '-d' command-line switch for a daemon mode (angel process)
• added a '-k' command-line switch to gracefully stop gwan (daemon)
• added a '-r' command-line switch to run C scripts (like the ab.c test)
• escape_html() uses unicode for 8-bit chars and respects word breaks
• xbuf_xcat() supports the ' pretty thousands (%D, %U, %F still work)
• xbuf_xcat() supports the '#' flag (decimal point with no decimals)
• xbuf_xcat() supports %b to print 16-bit, 32-bit and 64-bit binary ints
• servlets life is now limited by a time-out value (no more infinite loops)
• servlets that fail return '500:Internal Error' rather than '404:Not Found'
• fixed: files deleted on disk can no longer return the next cache entry
• fixed: removed a stupid limitation for long servlet output (loan100+)
• fixed: I have had a hard time to identify/fix this system socket issue
• the ab.c benchmark framework now dumps HTTP errors (IIS locks)
• factor 4 speed gain (the kernel-bottleneck will let you see only 2% of it)
• factor 5 scalability gain (hopefully, you will see 100% of this clever one)
• factor 20 (peak value) of CPU resources savings (high-concurrencies)
• high-concurrency attacks on long-lasting scripts are no longer harmful.

Upgrading G-WAN/Windows to v1.0.6 would not make it as fast as on Linux: the Windows kernel has hit the wall (see notes [2] & [3] and Linux vs. Windows). As long as Microsoft does not fix its kernel, it does not make sense for TrustLeap to invest more time on Windows because TrustLeap's goal is to offer the most efficient solution for the benefit of developers and end-users.