User defined macros

I have just finished hacking in user defined macros. For the moment they look like they will be up to any use I want to make of them at the moment. They are fairly simple, they are syntactically similar to functions. They must return a list. This then replaces the invoking code during compilation.

Back from beyond the grave

Hello.

Long time, no post. I don’t really have an excuse for that. Just generally being busy and not having a huge amount of time for personal projects. Hopefully I will get a little time over christmas.

I’ve done a little here and there, but nothing that seemed worthy of an update.

Today, I have started back into my Lisp interpreter. I fixed a load of minor bugs. I’ve added small amounts of other functionality. For example, I have added a type, similar to Lua’s lightuserdata. To test it, I made a set of functions for file manipulation. That, combined with the eval function I already had allows me to define (purely in Lisp) functions for including files.

I am an idiot.

I have just figured out where I am going wrong. All my variables are stored on the LISP stack, which is a std::vector<Cell> instance, (Cell is the variable type). Some of these Cells themselves contain Standard C++ Library objects (lists use std::list<>, functions are std::vector<Instruction>, strings are std::strings), and references to those objects are stored on the C++ stack during operation.

A few functions deep, when stuff keeps being added to the stack, it eventually will reallocate and… boom, all my iterators and references die a horrible death. Took quite a while to figure out that this was going on, I was preoccupied with assuming there was an issue with the Cell’s implementation of the Rule Of Three. I had already had issues with that, as I mentioned before.

For the moment, a “quick fix” is to manually set the initial stack size quite high. I think with this my current implementation works fine, but I hate such an ugly fix, which is too brittle anyway. I could choose std::list<>, so elements won’t be moved around in memory, but there might be another way.

That took far too long to diagnose…

Quick update

I’ve been pushing off writing the update for some time, being both too busy to finish my implementation and plagued with issues with the code as it is right now. In an effort to move me forward, I will write a note that I hope will motivate me to action.

I am very close to completing a large re-organisation of the codebase. I think the only major outstanding bug is that my call stack is being corrupted somehow. From inspection of the code, a function should remove all its arguments and push its result, but somewhere (1 place in particular) this isn’t occuring. While debugging it, I decided that this might be a good opportunity to throw together a quick debugger for my new language, but currently only implemented at the Interpreter “byte code” level. It allows me to step through, and over code, to inspect variables and to examine the contents of the stack. This gives the advantage that I can skip large bodies of C++ which have nothing to do with the problem when it is a “high level” issue, rather than a bad pointer or something.

On the subject of bad pointers, I fixed a bug with my variable class. Apparently my assignment operator made the assumption that the old and new objects shared the same type. Obviously, some serious pointer corruption issues occur when calling delete on standard library objects throught the wrong type. I really can’t believe that I missed that.

Since I am finished working tomorrow, I should have more time to get this thing working. I have some ideas once I get to that stage to start doing some things in this langauge that aren’t in LISP (AFAIK, the poor language has how many illegitimate children?).

Nesting Quotes

Its been slow going, adding small bits here and there. I’m most of the way there, I just need to re-integrate the macro handling system back into the compiler. I have added a preprocessor that replaces the quoted literals like ‘X with the macro (quote X). This allows nested quoted literals to survive the parsing process, which didn’t happen before. Before, a quoted literal meant the parser entered a different mode whereby it only scanned lists, atoms, numbers and strings. This resulted in a long literal. While nested lists could be present, there was no way to tell if these nested lists were originally quoted.

One step forward, X steps backward.

I’ve made some progress, but I think I’m after hitting a wall I didn’t anticipate. For your consideration, I will first post what I have working, then I’ll briefly discuss what isn’t and why I’m going in the wrong direction.

More work.

Short update.

I’ve gotten local variables and function arguments to work, along with introducing a boolean type. And lambda. However, I’ve seen some serious flaws in how my current system handles complex literals. This impacts any code that will depend on them, such as versions of defun and if which don’t take strings as arguments. To counteract this, some serious restructuring of my parser may be required, and a lot of rethinking done. I spent between 4 and 5 hours today on a bus, so I have a few A4 pages full of notes and ideas that I think I might be able to manage. I have already begun to implement some of it.

What I have done now is made a complete “guess the number” game that I want to be able to compile. Over the next few days I’ll try my hardest to focus on features that are necessary to make it work. When that is done, I might write up a post on however the system works as of then. Looking over the last few entries, I am not happy with them. They are too focused on what I have working, not how I got them working – which is what I intended to write about. In the mean time though, I need sleep.

Conditionals and Functions, Oh My!

Ok, with a bit of effort (and excessive quoting) I have conditionals working, and LISP functions. Until I make the parser programmable, any expressions that need delayed evaluation need to be enclosed in strings. The relevant functions (if, defun) simply use the same internal logic as eval does to evaluate the selected expressions. Currently, functions cannot take arguments, though they may return results. Recursion also appears to work!

More LISP

Well, not really.

But I’ve come close:

(print (eval (readline)))

I’ve done a lot, although some of the big milestones that make a language useful aren’t finished.

Implemented to some degree:
* math functions
* list literals (I’ve fixed everything I could find with them)
* strings in quotes “”, which allow spaces and common escape characters
* multi line comments (again, lua style –[[ ... ]])
* eval!

Eval is important, because using it I think I can “fake” control structures like if, while by using quoted arguments. I may even be able to implement defun. However, that would be a stop gap solution, one that I imagine would be very slow. Currently I am thinking of quoting the arguments to if and while, and just interpreting them on the fly.

It has been a while…

A long while. Sorry. I have two excuses, a) I have a job and b) I haven’t done much programming lately.

Yesterday I have started a new project. I have been following a number of GD.net journals of late, and a number of users are making their own languages. This is an area that always fascinated me.

So my new project is such a language! Well, its not my own language – yet. At the moment I am using a syntax heavily inspired by LISP. Its been a while since I’ve used LISP, so some of the details might be different. The main reason for this is because I wanted to try my hand at writing my own parser. So far, its been pretty easy, with two notable exceptions. The first is the most pressing, error handling. Right now I have no way of reporting to the user which line a syntax error occurred on. The second is a minor one, list literals. At the moment, the parser will deep scan a list literal and find non literals inside it, whereas my intention is that it should eventually only find number, string and nested list literals inside a list literal. It shouldn’t be too hard to fix the latter, but I may have to make some larger changes to make a clean way of implementing the former.

For the moment, the actual runtime is coming along very well. I’ve gotten the following to work nicely: