There's a lot more going on than adding nodes to a linked list. The parser is parsing thousands of incoming course plots (a fine benchmark of parser speed) and the script is retrieving the parsed courses from the database (a heavy JNI workload).

Yes, the CPU load on the script side is minimal, just iterating over courses and setting bits in a BitSet. But I highly doubt any script for any helper is CPU-bound rather than IO-bound.

Except parsing courses isn't exactly difficult, either.



IO is in the microsecond range. Compared to ping,
that's irrelevant.

I can easily tax the processing power of TWX proxy
and make scripts CPU-bound. Very very easily.

When people run comprehensive benchmarks of
software or hardware, do they just run one test
or do they compare multiple methodologies and
contrast their strengths and weaknesses?

It's probably the most CPU-intensive thing a helper does.



IO includes network and disk access. These are the main things limiting any script. Not CPU.



I doubt that. It would have to be a pretty contrived benchmark to single out the CPU, i.e., working on a large amount of data in RAM without making any use of the database or the network. What scripts have you written that do that? About the only thing I can think of would be a bubble finder or some sort of traffic analysis. (If the entire database is cached, it would eliminate disk IO... but even then, SWATH is hindered by JNI and puts in a good showing despite that.)

What kind of script do you think would make a good benchmark?

Parsing courses is no where near as CPU-intensive as actually calculating them from a map.



IO is disk access and RAM access. Network speed is latency. IO is not a big problem in my scripts. CPU definitely is.



Bubble searches, traffic analysis, optimal course searches, array sorts, etc, etc.



Generally, yes, I cache the database into a set of arrays. In fact TWX proxy automatically caches the entire database if you let it.



I think a wide variety of scripts would be a better approach. ZTM, world SST, bubble stuff, traffic analysis, sorts, etc. If the goal is to honestly compare the speed of both, why not do exactly that?