Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[kytuzian]

Urgh, I'm stupid. Sorry about that.

Code: [Select]

(repl (new-poslin *prim*))
I downloaded CCL and after doing some fruitless bug tracking I realized what the problem was and tried this. It works on my computer.

The macro `new-poslin` takes a list of variables containing a list of primary operation descriptions so these can be defined. So, if no primary operations actually are defined, it's no wonder when calling `!` doesn't do anything – there is no such operation and it's not immediate yet.

Once again, I'm sorry for that.
I don't know about that warning, it seems inconsequential. If I knew where it was coming from, I might be able to fix it, but there's no hint at where that lambda is coming from or anything, so…

A word of warning: CCL will throw an error about exhausting the stack or something like that when you QUIT the repl while a sufficiently large environment is on the current stack. This seems to be because the printer runs out of memory trying to print all the stuff in the environment. If you don't want this to happen, run the REPL like this:

Code: [Select]

(progn (repl (new-poslin *prim*)) (values))
This returns nothing, so nothing needs to be printed.

Okay it works now, I've got work now but when I get home I'll start messing around with it. Thanks for all your help.

[Antsan]

I merged the readable branch to default, so if you compile from that, the old standard library will not work.
The part of the standard library that has been done has been pushed to the poslin-lib repository. The standard library is the base.poslin file in the toplevel directory, as usual. The other libraries are outdated and should probably be removed, now that I think about it… and so they are now.

As planned before, there will be several types of setting operations.

"-set" is the most basic. It doesn't care about bindings, so if the value to be stored should go into an environment, it needs to be a binding, as environments only contain bindings.

"-write" first stores your value in a binding and then does the same as "-set" with that binding.

"-ensure" is like write, only that overwriting anything is avoided – so, if you do

Code: [Select]

foo [ some stuff to do an operation ] op-ensure !
foo is only defined with the given definition if it wasn't defined before.

Another kind checks for an already existing binding and replaces it's contents if it is already present, otherwise it works like "-write".

The last kind only writes into already existing bindings and raises an error otherwise.

Any ideas what postfix to use for the last ones names? Maybe "-overwrite" and "-rewrite"?

[Antsan]

Once again, I have naming problems.
As you might remember, defining operations can be done like this:

Code: [Select]

2+ [ 2 + & ]o
That last symbol is the problem. There are 5 ways to write into an environment, 4 of which are applicable for this situation – I might want to define the operation with a fresh binding, so previous uses of that operation aren't affected or I might want to use ensure to define an operation only if another implementation isn't already present or I might want to redefine a previously existing operation so that others who use the binding to call the operation use the newer version or I might even want to make sure that I am overwriting something. All of those would need a different variation of `]o`. How to name them?

Code: [Select]

-write     ]o*  because '*' is for "new" and everything about this definition is new?
-ensure    ]o?  because the definition might or might not be used?
-overwrite ]o   because this is probably the version you would want to use most of the time
-rewrite   ]o&  because… I don't know. Better ideas for this one? Anybody?


[kytuzian]

Alright so I said I was going to start looking at stuff a little bit ago, turns out, that's today.

Anyway, I can do some basic stuff, but I'm pretty confused about how this language works.

• What are the 'var', 'op', and 'stack' thingies by operation definitions in the square brackets. I assumed that they specified the type of the arguments?
• How do I get the length of a stack, or otherwise iterate over it?
• Can I use operations as arguments, and if so, how?

I tried doing something like this:

Code: [Select]

do-op [ ]
{
    op arg~
    b arg~
    a arg~

    a get b get op get & &
}

but it just puts the arguments back on the stack, it doesn't actually do the operation.

[Antsan]

Alright so I said I was going to start looking at stuff a little bit ago, turns out, that's today.

Anyway, I can do some basic stuff, but I'm pretty confused about how this language works.

• What are the 'var', 'op', and 'stack' thingies by operation definitions in the square brackets. I assumed that they specified the type of the arguments?
• How do I get the length of a stack, or otherwise iterate over it?
• Can I use operations as arguments, and if so, how?

1. These are not for argument types but tell the operation defined which local scopes should be used. `var` opens a lexical variable environment when the operation is called, `op` a lexical operation environment and `stack` lets the operation use its own stack. So, if you want to define variables that are only visible inside the operation you are defining, you need to put a `VAR` into those square brackets.

2. There is nothing like that in the standard library. I assume you already know what mapping is. Here is an operation or mapping over a stack, using the old operation names:

Code: [Select]

map [ var ]  ;[ `map` uses local variables (the arguments) and thus needs its own lexical variable environment ];
{ op arg~ stack arg~  ;[ remember the implications of postfix notation regarding in which order arguments are taken from the stack – I will try to rectify this with the `[ arg1 arg2 … ]args` construct ];

  stack get [] ! == &  ;[ check whether the stack is empty ];
  [ [] !  ;[ if it is, we just return the empty stack ];
  ]#  ;[ otherwise: ];
  [ stack get _ & op get map call  ;[ first map over the rest of the stack ];
    stack get : & op get call  ;[ then call our operation on the top element of the stack ];
    <- &  ;[ then push the result of that back onto the mapped rest of the stack ];
  ]#
  ?  ;[ decide which of the two branches should be called ];
}
3. Yes, you can. See below.

I tried doing something like this:

Code: [Select]

do-op [ ]
{
    op arg~
    b arg~
    a arg~

    a get b get op get & &
}

but it just puts the arguments back on the stack, it doesn't actually do the operation.

Always keep in mind that, when an operation is defined, there is already poslin code running that, in a way, assembles the operation you are defining.
Specifically this is about immediate functions.

Now your code has two issues, one unrelated to the problem you are describing. That unrelated problem is that this operation does not use its own variable environment, so it will instead just write your arguments into the variable environment in which you are calling this operation. To avoid that, place a `var` into the square brackets behind te operation name.

The other issue lies in this specific chunk of code:

Code: [Select]

op get & &
This contains two different immediate functions, `get` and `&`. These are called as soon as they are read.
`get` dumps a thread onto your current stack (which will later be transformed into a thread itself). his thread is responsible for retrieving the value of `op`.
`&` takes an object and transforms it into a thread. That means, in this special case, the first `&` takes the thread `get` dumped and then transforms it into a thread, which is a no-op. The second `&` again takes that thread and then puts it back, because it already is a thread.
What you want is to place a thread here which calls the object on top of the thread. The corresponding operation is `!`. Now, keep in mind that `!` is immediate but you don't want it to be called now but rather you want its thread. You need to quote it so the symbol `!` ends up on your stack and then transform that symbol into a thread using `&`. So the code chunk should look like this:

Code: [Select]

op get '! &
This is a bit ugly, so there is another immediate operation, which does that for you, called `call`.

Code: [Select]

op get call ;[ as seen in the implementation of `map` above ];

So, your operation actually should look like this:

Code: [Select]

do-op [ var ]
{
    op arg~
    b arg~
    a arg~

    a get b get op get call
}
If you've got any questions, I am happy to answer them. I need to learn explaining this stuff either way – after all, if I want this to be used, I'll have to write a tutorial.

[Antsan]

poslin0.1.0pr5 has been uploaded.

This release uses readable operation names and includes these new operations:

Code: (fold-stack – folds a stack, equivalent to Haskell's foldr.) [Select]

fold-stack [ var ]
{ op arg~ null arg~ stack arg~

  stack get
  [] same? &
  [ null get

  | stack get drop &
    null get op get
    fold-stack call

    stack get top &
    op get
    call
  ]if
}


Code: (map-stack – map over a stack) [Select]

map-stack
[ '[ & parent-pop &
       call #
       push & #
  ']& &  ;[ I think this might be made more readable by using `]#` ];
  r<- &
  [] r-> &
  fold-stack &
]o


Code: (map-environment – map over an environment. This modifies the bindings in the environment, so watch out what you do with this. This is intended to be used later with thread optimizing operations.) [Select]

map-env [ var ]
{ op arg~ env arg~

  env get env-symbols &
  dup-here &
  [] same? &

  [ [ pop &
      env get swap-here &
      env-lookup &
      op get modify &
      dup-here &
      [] same? & not &
    ]& do &
  ]unless

  drop-here &
  env get
}


Code: [Select]

]if
[ ']# & r<- &
  '[ & parent-pop &
       '& & '# &
       r-> &
       <?> & #
       call #
  ']& &
]i


Code: [Select]

]when
[ ']# &
  '[ & parent-pop &
       . & # #
       <?> & #
       call #
  ']& &
]i


Code: [Select]

]unless
[ ']# &
  '[ & . & # #
     parent-pop &
     <?> & #
     call #
  ']& &
]i

I did not change the standard library to respect already existing bindings for operations. That means, if you, by some kind of voodoo magic, happen unto a poslin implementation providing native implementations of some of the operations in the standard library, they will be replaced by the slow ones implemented in poslin itself. As it is quite unlikely that anyone but me provides a poslin implementation, I think this is not yet a problem.
Also `do` and `when` haven't been replaced by `]do` and `]when` (which aren't in yet), as they should be.
`?` also is still in the standard library, despite being obsoleted by `]if`. It will be removed later. The same goes for `when` and `unless`.

[Antsan]

I finally reimplemented the standard library. It's way cooler and cleaner than the previous version and it does check for already existing bindings before writing new definitions. As I am behind a terribly slow connection I won't upload a new distribution, but you can simply replace the base.poslin file in the last distribution with the new one from the repository.

Currently there are 65 primary operations and 221 operations defined in the standard library, giving you 286 operations to work with, which is way too much to tel you all about here.

But some things I can highlight.

Let's begin with the totally awesome `<<`:

Code: [Select]

'<<
[ '[ & r<- & #
       parent-pop &
       r-> & #
  ']& &
]i?
This is an immediate operation to be used when you want an operation to manipulate something further down the stack. It takes the previous operation and wraps it so it operates one item further down the stack. Because that wrapping is done at compile time, you can stack them.
So, instead of writing the tedious

Code: [Select]

r<- & r<- & r<- & do & stuff on & the & fourth & item on & the & stack & r-> & r-> & r-> &
you can now write

Code: [Select]

[ do & stuff on & the & fourth & item on & the & stack & ]& << << <<
which is much shorter, more readable, always balanced and easier to correct if you miscounted.

New slots containing environments and all the associated operations (getting and setting the environment itself, getting at setting in the environment) can now be defined with `def-env-slot`. It expects first the prefix for all the operations and then the name of the slot itself, like this:

Code: [Select]

foo FOO def-env-slot
The `def` can be used like this (which will write into existing bindings or create new ones of they aren't present) or postfixed with ?, *, + or / for another policy for handling existing bindings. What these do exactly, see below. It looks like this:

Code: [Select]

scope SCOPE def?-env-slot
Please note that this only influences how bindings of already existing bindings for slots and operations are treated. It will still define all of the setter types.

There are 6 setter types. They all differ on how they treat existing bindings or whether they deal internally with bindings at all.
The simplest setter is *-set (replace the star with op or env, for instance), which expects a binding and just writes this into the respective environment.
*-write creates a fresh binding for you and saves that. (the * postifx)
*-overwrite creates a fresh binding if none is present and otherwise writes into the existing binding. (no postfix)
*-rewrite writes only in an already existing binding and throws an error if no binding is present already. (the / postfix)
*-ensure only writes a fresh binding if none is already present. If one is present, it does nothing. (the ? postfix)
*-establish only writes a fresh binding if none is already present. If one is present, it throws an error. (the + postfix)

`do` has been discarded and `while` has been replaced with `]while`.
Use it like this:

Code: [Select]

[ condition for & running the loop &
| loop body &
]while

If you define an environment slot and think a lexical scope for it should be present, you can do it with this:

Code: [Select]

foo FOO def-std-env-scope
The first symbol is the one you have to supply for the operation definition, the second symbol tells Poslin which slot should be affected.
So, a lexical foo environment can then be used like this:

Code: [Select]

bar [ foo ]
{ do stuff with foo local to bar
}

Well, I think this is the most important stuff. If you want to know which operations are defined at a certain point, just run

Code: [Select]

op-env ! env-symbols !
This will give you a stack with all the symbols with definitions in the current operation environment.

[Antsan]

@kytuzian: Do you use Clozure CL or Corman Common Lisp? I think I might be able to fix the problems with REPL-DYN on non-sbcl common lisps.
Are you familiar with folding? Using "fold-stack" you can calculate the length of a stack like this:

Code: (If you want to see the length of a stack right now) [Select]

0 [ drop-here & 1+ & ]& fold-stack !


Code: (If you want an operation that calculates the length of a stack) [Select]

stack-length
[ 0 [ drop-here & 1+ & ]# fold-stack &
]o

You could also use 'while':

Code: [Select]

stack-length [ var ]
{ stack arg
  length 0 let
  [ stack get empty-stack? & not &  ;[ stop iterating when the stack is empty ];
  | stack drop ~ var-mod &  ;[ the `~` prepares `drop` to be called by `var-mod`. ];
    length 1+ ~ var-mod &
  ]while
  length get  ;[ return the length value ];
}

Turns out you cannot just overwrite the old base.poslin with the new one. Because primary operations like "string<-" and "prim<-" looked strange I renamed them to ">string" and "->prim" and the like.
Also primary threads have been renamed to elementary threads, as their most important quality is not that they are primary but rather that you cannot take them apart. So, "->prim" has been renamed to "->elem-thread".
I hope to get a new distribution up on Sunday.

"->elem-thread" takes a string and an object. It then leaves an elementary thread which does what the thread it consumed did and has as name the string you supplied.

Also there were some bugs in that standard library.
"slot-ensure" would leave junk on the current stack when the slot to be defined already was defined.
"def-env-slot" and family defined "*-env-binding" and "*-env" wrong, so they returned the binding/environment which was valid at definition time.

The FEATURES slot I almost forgot about now is properly updated.

Lastly I added new functionality agains: compilation hooks. The operation "&+" retrieves the "compilation-hook" value from the environment in the "HOOK" slot and calls that after calling "&". Currently the thread stored there is just the noop. Later it will contain an operation for optimizing the freshly created thread, but that will go into its own library.

I forgot last time: There now is the immediate operation ']args. Instead of writing

Code: [Select]

foo [ var ]
{ b arg a arg
  …
}
you can now write

Code: [Select]

foo [ var ]
{ [ a b ]args
  …
}
which not only is shorter but also more visible and has the parameters in the order you would see the arguments on the stack.

What was "call" before is called "dyn-call" now.
"call" now retrieves the binding of the operation and inserts code to get the value of that binding and call it. Together with "declare" it can be used for recursion. It's faster than "dyn-call" and doesn't rely on the operation-environment at runtime.

Code: [Select]

foo declare
[ dup-here & positive? &
  [ dup-here & ->string & print &
    1- & foo call
  ]when
]o

The standard library is up to 244 operations, giving a total of 309.

[Antsan]

Characters where missing from Poslin. I added them now. Poslin uses unicode characters and you can enter any character in that standard with it's name like this:

Code: [Select]

<latin-small-letter-a>
It just shouldn't contain any whitespace, because that is taken to be a token border, so you'll get multiple symbols instead.
Any single character can also be entered:

Code: [Select]

<a>
The REPL will still print it back to you with its full unicode name.

Consequently the operation <?> had to be renamed. It's now simply ?.

The operation char->int transforms a character into an integer (which should be its Unicode code-point, unless I did something wrong), int->char is the reverse.

Array operations don't work on strings anymore. How strings are implemented is up to the implementor.
For strings now there are the operations string-lookup and string-set.

69 operations in poslin0, 313 in poslin1, still 244 in the standard library.

[Antsan]

New distribution is up.

Next I probably should clean up the implementation itself. Error handling is kind of hacky right now.
I'm not going to implement something like throw or catch in the basic error handling, though. I think that can be done in a library with… uhm… return stack markers?

Basically a throw would start taking stuff from the return stack, matching against some constant value, the "catch" marker which had been pushed onto the return stack earlier in execution.
That value would need to disappear silently if the throw never happens. The one value with the property of not doing anything on the return stack is the noop. As different throw statements need unique markers, we can't just use noop itself (even more so since I am not sure whether it is possible for a noop to end up on the return stack during normal execution).
So, we need a way to make new markers.

Code: [Select]

new-marker ;[ string ];
[ . ~ ->elem-thread &
]o?


[Antsan]

I implemented two optimizations, although they will produce incorrect behavior in some cases. Although I tried to account for return stack manipulation (which is what I tried to optimize in the first place), I didn't check for anything but the most obvious problems.

The first optimization is to flatten threads, so that less jumps happen (also known as "inlining"). The idea is to transform something like this:

Code: [Select]

{ { #{1} P{+} } P{*} }
Into something like this:

Code: [Select]

{ #{1} P{+} P{*} }
You might see why I call it "flatten".
Just naively flattening everything would lead to problems when doing it with a thread like this (I'm too lazy to replace the C{drop-here} with the actual thread and it wouldn't tell you much either way – what this does is throwing away that thing just popped off the return stack, which always would be the part of the thread displayed as '…')

Code: [Select]

{ P{+} { P{r->} C{drop-here} } … }
If we just flattened this, this thread would do something else entirely. So I look how much of the return stack might be thrown away and only flatten so much that the thread in question stays nested deep enough to not change its meaning. As 'r<-', when unbalanced, might push any thread onto the return stack (unless I do some more serious analysis) which might manipulate the return stack in unforeseen ways itself any thread which contains an unbalanced 'r<-' at all isn't flattened (although subthreads might be flattened).
Unfortunately my implementation does not look into constant threads, because I'm not sure how to handle the knowledge I might acquire. Also unfortunately every sensible use of 'exit' would be found inside a constant thread returning a thread. Personally I don't use exit, so it's not a problem with the standard library, but that's hardly an excuse to not do this properly. I'll save this for a later date, when I have an idea of how to process threads comfortably.

The second optimization, which relies on the first one, is to remove subsequent calls of 'r<-' and 'r->'. This one is pretty straightforward and will always work correctly (but it might receive already incorrect input due to incorrect flattening). It might be made more thorough, but that's a lot of work. First I should correct that first optimization step.

Here's the implementation:

Code: [Select]

thread-balance declare [ var ]
               ;[ takes a thread as input and leaves an integer and a boolean as
                  return values ];
{ [ thread ]args
  ;[ it is necessary to know what the balance overall is – if it happens to be
     positive, the thread is not balanceable ];
  overall 0 let
  ;[ if 'lower' goes below 0, the thread might not be neutral on the return
     stack that far down ];
  lower 0 let
  ;[ how much inner complex threads might pop off the return stack ];
  inner-lower 0 let
  ;[ if an inner thread isn't flattenable, this one isn't either – this is only
     about threads leaving something on the return stack ];
  flattenable? .true ! let
  ;[ the part of the thread we are currently checking for its influence on the
     overall balance ];
  curr .nothing ! let
  [ thread get . ~ same? & not &
  | curr thread get thread-front & set
    curr get :Thread type? &
    [ curr get thread-balance call                  ;[ check balance of complex
                                                       subthread ];
      flattenable? << flattenable? get and & set    ;[ this thread is only
                                                       flattenable if every
                                                       complex subthread is ];
      inner-lower << inner-lower get minimum & set  ;[ if the lower balance of a
                                                       subthread is lower than
                                                       this one, update ];
    | curr get r<- ~ same? &
      [ overall 1+ ~ var-mod &                      ;[ r<- has a balance of 1 ];
      | curr get r-> ~ same? &
        [ overall 1- ~ var-mod &                    ;[ r-> has a balance of
                                                       -1 ];
          ;[ if the overall balance is negative, check whether it is lower than
             the lower bound up to now and update, if appropriate ];
          overall get negative? &
          [ lower lower get overall get minimum & set
          ]when
        ]when
      ]if
    ]if
    ;[ update the thread variable ];
    thread thread-back ~ var-mod &
  ]while
  ;[ return the lower of the inner lower bound+1 and the own lower bound of
     balance. ];
  lower get inner-lower get 1+ & minimum &
  ;[ if the balance is positive or any subthread isn't flattenable, this one
     isn't flattenable ];
  overall get positive? & not &
  flattenable? get and &
}

thread-onto-stack  ;[ transforms [ … ] { … } into [ … … ] ];
[ [ dup-here & . ~ same? & not &
  | dup-here & thread-back &
    [ thread-front & push & ]& <<
  ]while
  drop-here & ;[ get rid of the stry noop left by iteration ];
]o

thread-flatten-toplevel [ var ]
{ [ thread ]args
  ;[ push onto a stack for processing – it's much easier than trying to
     construct a thread directly ];
  result [] let
  [ thread get . ~ same? & not &
  | thread get thread-front &
    dup-here & thread-balance &
    zero? & << and &  ;[ if the thread is flattenable and it's balance 0… ];
    [ [ result result get ]& <<
      thread-onto-stack &
      let             ;[ …then push the thread contents onto the result ];
    | [ result result get ]& <<
      push &
      let             ;[ …else just push the thread as whole onto the result ];
    ]if
    thread thread-back ~ var-mod & ;[ update thread variable for iteration ];
  ]while
  result get '& &  ;[ return the result and translate back into a thread ];
}

;[ this one looks a bit weird, yes, but it works as intended (not counting
   incorrect balance analysis) ];
thread-flatten declare
[ [] << thread-onto-stack &  ;[ prepare the thread for mapping ];
  ;[ now map over the stack representing the old thread like this: ];
  [ dup-here & :Thread type? &  ;[ if the thread we're dealing with is
                                   complex… ];
    [ thread-flatten call       ;[ …then flatten its subthreads ];
    ]when
    thread-flatten-toplevel &   ;[ flatten the thread itself ];
  ]# map-stack &
  '& &                          ;[ turn the result into a thread ];
  thread-flatten-toplevel &     ;[ flatten it ];
]o/

clean-rmanip declare [ var ]
;[ clean the toplevel of the thread of noops and redundant return stack
   manipulation ];
{ thread-flatten &
  [ thread ]args
  ;[ this is done by iterating over the thread, keeping tabs on the first two
     operations not already processed. If they have opposite stack effects,
     don't move them to the result. If the next to process is a noop, don't move
     it to the result either. ];
  result [] let
  last . ~ let
  this . ~ let
  o( step
     [ last this get set
       this thread get thread-front & set
       thread thread-back ~ var-mod &
     ]o
     [ last get . ~ same? &
       this get . ~ same? & and &
       thread get . ~ same? & and & not &
     | step &
       last get . ~ same? &
       [ last get r<- ~ same? &
         [ this get r-> ~ same? &
           [ step &
           | result result get last get push & set
           ]if
         | last get r-> ~ same? &
           [ this get r<- ~ same? &
             [ step &
             | last get :Thread type? &
               [ last clean-rmanip ~ var-mod &
               ]when
               result result get last get push & set
             ]if
           | result result get last get push & set
           ]if
         ]if
       ]unless
     ]while
  )_
  result get '& &
}/

Now all operations defined until now (unless they're hidden in a binding in a thread somewhere and inaccessible otherwise) can be optimized via:

Code: [Select]

> op-env ! clean-rmanip & map-env ! drop-here !
This won't optimize inside constant threads (as you already know) and break everything relying on strange return stack magic.

[Antsan]

A ']for' operation is in.

Code: [Select]

[ initialize-variables &
| step-variables &
| continue? &
| loop-body &
]for
The specific layout does not reflect the order in which these things are done. Variable initialization, of course, happens first. The check for looping happens before the loop body. Whatever step-variables is happens after the loop body runs.
So this:

Code: [Select]

[ v 1 let
| v 1+ ~ var-mod
| v get 10 <= &
| v get ->string & print & newline print &
]for !
prints this:

Code: [Select]

1
2
3
4
5
6
7
8
9
10
The for loop uses its own local variable environment.

var-mod has been changed to a macro, that is, it is an immediate symbol now and puts the associated thread on the current stack.

Code: [Select]

var-mod
[ var-lookup & << modify &
]m?

reverse-stack has been added.

[Antsan]

A new release (version 0.2.1) is up, this time for linux-x86. I'll need to get used to updating the version number in the repl welcome message and naming my uploads consistently.
A cautionary tale accompanies it. (TLDR: Loading files is much faster now and letting the computer recompute the same values over and over is stupid)

If you ever ran poslin1, you probably noticed the long startup time. I thought that this was due to the standard library being run.
I was utterly wrong. Once the standard library is read, it runs almost instantly. Inserting a few print operations into it showed me that almost all of the time was spent in parsing.

You see, I use cl-ppcre for that purpose, which is an implementation of perl-style regular expressions, claimed to be more efficient than Perl. (I don't actually use the Perl syntax, though, I prefer to have my regular expressions in tree form)
cl-ppcre is really nice for scanning a string for a given regular expression. It does not provide a way to automatically extract the first match of several regular expressions together with information which regex matched, which is what I need for tokenization. So I wrote that myself, by just finding the first matches for all token types, cutting them all out of the string (in the case of overlapping matches I had some rules I forgot right now on which was to be taken, but in the end it works out to take the token first found and the most… uhm… usable one when multiple matches began at the same point), registering them together with their boundaries inside the string, repeating until I didn't find anything anymore in the string, then sorting by their beginning positions and, converting into poslin objects with the information obtained and that's it.

A few days ago that scheme seemed ugly, so I replaced it with a different scheme where I would check for the first occurence of any token, get the one occurring first (preferring tokens associated with more specific regexes over the general ones – parts of a string can be matched by the regex for symbols), registering and converting that token, saving it into a list, cut off the consumed part of the string and then repeating that until the string was empty.
You might already see how this is a horrible idea. It was painfully, awfully slow, even slower than my first implementation.
Why? Well, suppose you have no floats at all in the string you're parsing. Now, for every token that is read, the parser again and again checks for the presence of floats inside the string, always processing the whole string.
In short: Work already done is repeated over and over. This also happened with the older scheme, only that there it only happened with tokens which weren't in the rest of the string anymore and not with any token type all the time. This scheme wouldn't recompute the position of the first integer over and over until reaching it but just cut it out immediately and then look for the next one.

After realizing that I did the right thing, of course, and only recalculated the found matches that weren't valid anymore after cutting out the first token. Any token type not found isn't checked for again as long as we're processing the same string and work isn't duplicated.
Now loading the standard library seems to be almost instant.

[Antsan]

Poslin 0.2.2 is up

• The `:String` type has been added properly now.
• The immediate `]string` operation has been added. It creates an operation which makes a string of the objects given. It supports threads taking no value and returning one, which will be called when the resulting thread is called. In a context where the variable `a` is defined as 1, `[ "The value of a is " a get ]string !` would return the string "The value of a is 1".
• The immediate `]message` operation is almost the same as `]string`, but prints the resulting string.
• Evaluation of blocks of code depending on the presence of a feature has been added. `FEATURE f[ code ! ]f` will only evaluate `code` if `FEATURE` is a defined FEATURE (in the FEATURES slot). `FEATURE -f[ code! ]f` is the opposite.
• Some bugs in the standard library have been fixed
• `get` has been changed so it immediately creates one thread that returns the value of a variable, so it can be more easily used inside string and message blocks

I think I didn't say anything about features before, so here's a quick overview:
One of the slots defined in Poslin0 is the FEATURES slot, containing a stack on which symbols are pushed. These symbols describe which features are available in the runtime.
The feature PRIM shows that Poslin0 is available.
The feature STD-LIB shows that the standard library is loaded.

You can add a defined feature with `register-feature` like this:

Code: [Select]

MY-AWESOME-LIBRARY register-feature !
You can check for the presence of a feature with `check-feature`:

Code: [Select]

[ ]
> PRIM check-feature ! ;[ this will always be defined ];
[ <TRUE> ]
> FOO check-feature ! ;[ assuming that FOO is not a defined feature ];
[ <TRUE> <FALSE> ]

The freshly added immediate operations `f[` and `-f[` are described above. The immediate operation `]f` is only defined in the environment opened by these two.

Remember that the operations described here are defined in the standard library. Without the standard library you have to check the FEATURES slot "by hand".

If you wonder why the `PRIM` feature is put into FEATURES in the first place:
Just imagine you have a local environment on the path which does not give access to many of the operations in poslin0, maybe because it is part of a programming game that lets the player play around in a safe sandbox poslin you created by making said local environment. That sandbox might not (I think even that it cannot, but I'm not quite sure) provide the operations of Poslin0, as those could be used to break out of the sandbox, so the FEATURES slot should not contain the PRIM feature.

[Antsan]

Poslin0.2.3 is up because I fucked up.

The definition of `dyn-call` used `<<`, which wasn't defined at that point, which didn't check with me as it is sparsely used. It is used in `fold-stack` and that one didn't work as expected, producing really strange results like this:

Code: [Select]

[ ]
> [ 1 2 ] 0 + & fold-stack !
[ 0 1 {P{r<-} P{+} P{r->}} 2 {P{r<-} P{+} P{r->}} ]
Yeah, that doesn't really make much sense.

I think before I go on with all of that optimization stuff, maybe I should do unit testing and a test suite for the standard library first. And structs, probably. Doing everything regarding data structures by hand can be quite a pita.