Yabasic 3

Recently, a number of things have been frustrating me about the way Yabasic currently works.

1. Yabasic is too slow. For example, Perl is often significantly faster than Yabasic, despite having a much larger and more complex implementation, and languages like Lua outperform Yabasic in almost every situation. I’m not saying that Yabasic needs to be faster than every other programming language implementation out there (although that would be nice!), but in light of its simplicity and small size it should certainly be more efficient than it currently is. So the virtual machine needs to be simplified and improved. Using a more flexible language than C (the language in which Yabasic is currently written) would facilitate optimisations and simpler, more powerful internal workings.
2. Parts of the language (symbol handling in particular) are a complete mess at present. That is simply the consequence of many changes and extensions being made throughout Yabasic’s history. There are parts of “symbol.c” which I still don’t fully understand, even after working slowly on Yabasic for over a year.
3. Building on point 2, I feel that it would be almost impossible to implement efficient, flexible associative arrays (and, hence, proper user-defined types and structures) in Yabasic as the language currently stands.
4. Yabasic lacks good garbage collection at present. Memory should be freed when it is no longer required, and—for aesthetic purposes, and practical reasons on some systems—all allocated space should ideally have been freed by the time the program exits (including when it exits due to an error). C++ facilitates this, I understand, with “destructors” and a cleaner memory management system in general. In C, all garbage collection must be done manually—something that is hard to do well and frequently risky and bug-prone.
5. I’m sure there have been other things that I’ve found quite problematic in the current C implementation, but I can’t recall them at present.

Therefore, Pedro and I have decided to rewrite Yabasic from scratch in C++. The overall syntax for, and interface to, Yabasic will not change much, if at all, in this process. And there will be other benefits to (hopefully) correcting the problems mentioned above when we switch to C++ as the language for Yabasic’s implementation: in particular, better organisation of Yabasic’s source code, which will allow us to find and eliminate bugs more quickly.

One of our main concerns is portability: we really want Yabasic to run on as many systems (even really obscure ones) as possible, while having a minimum of system-dependent code. This may mean that, for example, we need to write the “lexer” (scanner) and parser by hand; but, if we need to do this to make Yabasic compile and function properly on all kinds of systems, we will.

I realise you folks may find this setback a little annoying or frustrating, and I quite understand your feelings. I never expected development of Yabasic 3 to take this long myself! But I hope you can see that a change to C++ is the best road to go down now; and, if you don’t, we would like to hear from you (see below). It is my hope that, with both Pedro and I working on this, we will be able to get more work done in a shorter space of time than before.

If you really have an objection to this change, or if you support it, or if you are able to help in any way, then we would like to hear from you! Well, basically, we’d love to hear from you regardless of what you think. Pedro and I plan to run a real-time discussion about Yabasic on the #basicprogramming.org channel on Freenode (yes, there actually is an IRC channel for Basicprogramming.org!) sometime in the near future, and people will be free to ask questions, make suggestions, and discuss implementation details then.

Until then, feel free to comment on this blog or on the forums!

“Associative arrays” are just like normal arrays, except that they can be indexed by any type of value—a character string, for example—not just an integer. For example:

person ("Tom").place$ = "Australia"   // indexed by a string: "Tom"
person (1).place$ = "Australia"   // indexed by a number: 1

At the moment, I’m considering the possibility of re-writing Yabasic’s entire variable-handling system to improve speed and flexibility, particularly in order to facilitate the proper implementation of structures and user-defined types. If I went ahead with this, I’d probably be able to introduce associative arrays.

So what do you think? And if you’d like associative arrays to be implemented, what syntax would you prefer?

Recently, Derek reported a bug in Yabasic’s handling of subroutine recursion. I was rather worried by it, since such bugs are often extremely arcane and very difficult to locate.

Fortunately, I archive “snapshots” of Yabasic whenever I make significant changes to the code (yes, I should start using revision control software). As a consequence, when someone reports a bug that didn’t appear in Yabasic 2.763, pinning it down is usually just a simple matter of running through these archived snapshots until I find the version where the bug was introduced. Then I can compare the code between the working and the buggy version, and see what changes might have brought the bug into existence.

I must say I’m very glad that I do this; if I hadn’t, I estimate that I would have had to spend at least a day’s worth of work trying to track down the subroutine recursion bug that Derek reported. As it was, it only took me roughly half an hour.

The moral of this brief anecdote is this: if you’re rewriting a complex piece of software, continually archive versions of it every time you make major changes. That way, you’ll be able to find bugs that you introduce more quickly and more effectively.

Edit: Corrected embarrassing grammatical mistake.

A little while ago, Derek reported a strange bug. Code like the following, despite being in theory entirely palatable to the interpreter, causes Yabasic to spit out an error and exit immediately:

test ()

sub test ()
   struct items (1)
      item
   end struct
end sub

The exact error message is yabasic: program.yab:7: RETURN from a subroutine without CALL.

I suspect that somehow the “byte-code” generated by the structure is popping too many, or perhaps too few, elements from Yabasic’s internal stack. However, the bug hunt continues. Let’s hope I track the problem down this weekend.