Benchmarking Basic on Vintage Computers

There are a few machines I'm quite interested in comparing so I decided to create a simple Basic benchmark to get an idea of their relative speed. The benchmark tests 7 aspects and is inspired by qsbb, which is included with the qlay Sinclair QL emulator. The Basic source code for all the machines tested is available in the GitHub repo. I chose Basic because it was available on all the machines I wanted to test and because the versions I created for each machine would require few changes.

Benchmarks are only ever a test of what you are benchmarking and there are often other tasks that the machines would do much better at. In this case because all the benchmarks are written in Basic they are a test of both the hardware and the Basic implementation as can be seen by the Spectrum results.

Results

The results are listed below and for each the higher the number the faster they are.

Notes

Observations

The benchmarks for a few of the machines stand out for me because they were unexpected or confirmed what I had already heard.

Sinclair QL

The biggest surprise for me was how slow the Sinclair QL was. I knew it was hampered by the choice of a Motorola 68008, instead of a 68000, but the benchmark indicates just what a difference this decision made. It is particularly apparent if you consider the Amstrad PCW and Amstrad CPC6128 which were aimed at or could appeal to a similar market and deliver the sorts of speed that the QL should have delivered even if they had just stuck with the z80. That said, when it comes to the Functions and Maths test it was ahead of all the 8-bit machines and it would have been much easier to make use of the extra memory it could address without having to resort to bank switching.

Commodore 128

It is often said how slow CP/M was on the C128 and if you compare the benchmarks for the C128 vs the Amstrad PCW using Basic-80, it is quite clear just how slow it was. I'm also struck by how much slower the C128 was using Basic 7.0 in slow mode than the C64 using Basic 2. It would be interesting to know why this is.

Atari ST

The benchmark for ST Basic, while being top of the list, probably says more about the poor implementation of Basic than it does about the Hardware itself.

Sinclair Spectrum

This really underlines what a difference the implementation of Basic makes. If you take the Spectrum +3 running +3 Basic compared to when it is running Mallard Basic under CP/M; the Mallard Basic version runs about 5 times as quickly on the same hardware.

Emulators

All the tests have been run on emulators as I don't have all the machines and it makes the tests easy to reproduce. They all aim to be cycle accurate, but naturally they have to allow some flexibility otherwise they would be too resource intensive. Therefore the benchmark figures will never be quite as accurate as running on real hardware, but should give a good indication of their speed.

The Basic Code

I have put the Commodore Basic version below as I think it is the simplest version to understand. In the code the variable ti holds the value of the system clock and is updated every 1/60th of a second. The Basic source code for all versions are held in the GitHub repo.

10 rem basic benchmark
20 rem inspired by qsbb from qlay sinclair ql emulator
30 rem
40 rem copyright (c) 2019 lawrence woodman
50 rem licensed under an mit licence

100 rem array used later
110 dim ar(20)

200 print "basic benchmark"
210 print "7 tests, 20 seconds each"
220 print

1000 rem calculate benchmarks
1010 print
1020 print "please wait..."
1030 print

1100 rem benchmark printing
1110 t=ti
1120 p=0
1130 for i=1 to 20
1140 print ".";
1150 next i
1160 p=p+1
1170 if ti-t<=1200 then goto 1130

1300 rem benchmark functions
1310 t=ti
1320 f=0
1330 for i=1 to 20
1340 ra=sin(0.1):rb=log(4):rc=exp(10)
1350 next i
1360 f=f+1
1370 if ti-t<=1200 then goto 1330

1500 rem benchmark strings
1510 t=ti
1520 s=0
1530 for i=1 to 20
1540 a$="abcdefghijklmnopqrstuvwxyz"
1550 b$="zyxwvutsrqponmlkjihgfedcba"
1560 c$=a$+b$
1570 next i
1580 s=s+1
1590 if ti-t<=1200 then goto 1530

1700 rem benchmark for loops
1710 t=ti
1720 l=0
1730 for i=1 to 20
1740 for j=1 to 20: next j
1750 next i
1760 l=l+1
1770 if ti-t<=1200 then goto 1730

1800 rem benchmark gosubs
1810 goto 1830
1820 return
1830 t=ti
1840 g=0
1850 for i=1 to 20
1860 gosub 1820:gosub 1910
1870 next i
1880 g=g+1
1890 if ti-t<=1200 then goto 1850
1900 goto 1920
1910 return
1920 :

2000 rem benchmark arrays
2010 t=ti
2020 a=0
2030 for i=1 to 20
2040 ar(i)=i:b=ar(i)
2050 next i
2060 a=a+1
2070 if ti-t<=1200 then goto 2030


2200 rem benchmark maths
2210 t=ti
2220 m=0
2230 ka=5
2240 kb=6.2
2250 for i=1 to 20
2260 r=ka+kb-ka/kb*ka*kb
2270 r=5+6.2-5/6.2*5*6.2
2280 next i
2290 m=m+1
2300 if ti-t<=1200 then goto 2250

3010 print:print
3020 print "results"
3030 print "======="
3040 print "printing:  "; p
3050 print "functions: "; f
3060 print "strings:   "; s
3070 print "for loops: "; l
3080 print "gosubs:    "; g
3090 print "arrays:    "; a
3100 print "maths:     "; m