[z-machine] Tokenizing

[Andrew Plotkin]

On Fri, 18 Mar 2005, Mose Wingert wrote:

> For instance, the dictionary section of the Z Machine spec says that 
> dictionary words can start with non-alpha characters...they give "#record" as 
> an example...okay...so we can change alphabets within a dictionary entry. But 
> then does that imply that "#record" is distinct from "#Record" and "#RECORD" 
> in the dictionary?  Since we're trying to match user input, that doesn't seem 
> to make much sense...but I don't see anything to stop it in the spec.

As far as I know, yes, you can have capital letters in a dictionary word. 
Nobody *does* it, and I expect the Inform compiler refuses to generate 
such a word, because the player's input was lowercased when it was read. 
So your '#Record' word would never match the input.

But if you set up a line of text in memory, with capital letters, and used 
the tokenize opcode, then it would match dict words with capital letters.

--Z

"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*
I'm still thinking about what to put in this space.