[z-machine] Tokenizing

[Mose Wingert]

Glad to see there are other people out there still working with this...I've 
been having a blast writing my own Z-Machine, and have it nearly working on 
simple (v1 v2) story files.  But then I reached the read opcode...

I'm wondering if I'm missing something in terms of tokenizing...the read 
opcode says that the words in the input should be tokenized so they can be 
looked up in the dictionary.  But I'm not seeing anything to indicate how 
capitalization is handled...

For instance, the dictionary section of the Z Machine spec says that 
dictionary words can start with non-alpha characters...they give "#record" 
as an example...okay...so we can change alphabets within a dictionary entry. 
But then does that imply that "#record" is distinct from "#Record" and 
"#RECORD" in the dictionary?  Since we're trying to match user input, that 
doesn't seem to make much sense...but I don't see anything to stop it in the 
spec.

The only guess that I've come up with is that since the read opcode says 
that the text input will be translated to lowercase before putting it into 
the text buffer, that it means to do that for the parse buffer and 
dictionary as well...and that alphabets A0 and A2 are used in the 
dictionary, but A1 isn't...but that's pretty much a complete guess.

I dunno.  There's more I have questions about with regard to this, but maybe 
a lot of them will disappear if I get this figured out.

Many thanks for any help you can offer...

Mose