March 2023 Mutable

March 16, 2023

Strict is coming along nicely and all low level features are done, the early version of the VM is working, but needs some serious optimizations to be fast and useful for more complex problems. However currently it is still easy to add more language features or simplify some common programming issues, so I guess we are spending a bit more time on those things instead of finishing up.

Murali wrote about mutability last September already, this is a continuation of that blog post: https://strict.dev/blog/2022/09/20/details-on-mutability-in-strict

By default everything is constant

Last year we used "has" for members in types and "let" for constants inside method bodies. We already knew for years that mutable things are no good and want to code things as functional as possible. So "mutable" was used to describe when things had to be mutable, most of the time it was hidden away (like a for loop index counter).

To make things more clear "let" was renamed to "constant" because it will only compute a constant value or be used as a named inline. In method bodies you can use the "mutable" keyword instead of "constant" to make an actual variable you can change over time (and pass or return if needed). In all other cases "mutable" is not an available keyword, so instead you have to use the Mutable(Type) syntax, e.g. for return types or type members.

It is important to note that "mutable" code runs very different from normal code, many optimizations won't work and one of the main things that make Strict fast is not possible with "mutable": calculating once!

The Optimizer

Take this example:

    2 is in Dictionary((1, 1), (2, 2))

Here the Dictionary.in method is called (and tested, this test is actually from that method itself). On the left side we have a 2, which doesn't need any more optimizations or caching, it is just the number 2. On the right we are calling the Dictionary from constructor with a list of 2 lists containing 1, 1 and 2, 2. This will be done once and cached for a very short time (as the compiler will see no use for it after a few lines and discard it, the optimizer will already have checked for smiliar usages).

in the VM it looks like this:

  cached1122Dictionary.in(2) is true

Next up is the actual comparison if the in method returns true for the key 2, which returns true. If false would be returned, the program ends right here and the error is shown, we can't continue with any code path that leads here (TDD yo). So the VM can safely optimize this whole line away into: true, which clearly doesn't do anything useful and can be discarded alltogether (all unit tests work this way).

The same will happen for any other code of course. It is hard to show an example because useless code is not allowed and will lead to a compiler error unlike test code, which checks behavior and gets optimized away when execution is fine. But lets say we are looking at something like the Number.Floor method:

Floor Number
    1.Floor is 1
    2.4.Floor is 2
    value - value % 1

Again, here we can optimize both tests away as they are both true, but let's look in more detail how "value - value % 1" becomes 2 for the input 2.4. The VM will run the second test like this:

  Number(2.4).Floor is 2

which leads to the constant number 2.4 being cached

  value24 - value24 % 1 is 2

value24 % 1 is everything after the decimal point, so it becomes another constant number with the value 0.4

  value24 - value04 is 2

value24 - value04 is 2 (a bit cumbersome to read here, but this is all just optimization steps, nothing is actually computed or cached this way, it is just numbers), which makes the whole expression true and the test passes.

The point is that the compiler might find a faster or more efficient way in the future to calculate the floor of a number (without having to change any code) and nothing here changes. In fact nothing is even recalculated as all values were correct, only new code would be affected. Long term we want to keep results around in the VM, the strict packages and SCrunch, some of that works in memory, but since the VM is not done and optimized, not ready yet.

All non-mutable code is by default parallel, will be executed by as many CPU threads and GPU cores available and on as many computers as provided. Everything always runs asyncronly in the background and the compiler figures out himself how to put data back together if there were big loops or calculations involved.

Mutable

With mutable code most of these optimizations won't work at all or not in the same way. Let's look at an example from Number.Increment, method for mutable numbers:

    Mutable(5).Increment.Decrement.Increment is 6

The initial number 5 can't really be cached into value5 or something, as the very first operation makes it 6, then 5 again, then 6 again. We also cannot cache or optimize any steps in between as we can't know if the mutable outside of one of the mutable methods will mutate it further. Imaging running this in a big loop for every pixel of a 4k image, no good.

Here the compiler can obviously see that the final result of the last Increment is never used again as a mutable and convert it back to a constant and then roll back the whole calculation into:

    5 + 1 - 1 + 1 is 6

which again is true and can be optimized away. However the Mutable(Number).Increment method can't be optimized or used in the same as the Floor method above. The normal way to calculate a new number is of course not using Mutable, but just calculating the value with any operation you like and storing it somewhere else. Also using Mutable only in one place and NOT overwriting it for a long time can still be optimized very well (e.g. an image might be mutable overall, but within a method we will only set one pixel at a time, all pixel operations can be parallized).

Signals

Signals are used in many UI frameworks (javascript, typescript) and engines (godot) and are bascially the observer pattern but with UI elements. They are usually very cumbersome and hard to use, many game engines have something like this as well, but again, very specialized for ui elements or require to write a ton of boilerplate code to connect events (e.g. update ui label when hitpoints change).

I recently saw a very simple implementation and that got me thinking that this would be very useful for strict, obviously it requires a mutable thing, but since mutable is supposed to be used rarely, why not provide this feature for free: https://github.com/motion-canvas/examples/blob/master/examples/motion-canvas/src/scenes/signalsCode.tsx

The following code simply creates a radius 3 (mutable in strict) and then uses it for the area calculation (pi*radius squared). whenever you change radius, area automatically is updated (so internally each signal keeps track on where it is used and updates the outer signal as well, all event driven)

 const radius = createSignal(3);
 const area = createSignal(() => Math.PI * radius() * radius());

In strict:

mutable length = 3
mutable squareArea = length * length
squareArea is 9
length = 4
squareArea is 16

Simple, isn't it? also fits to our earlier ideas that all mutables become lambda calculations automatically anyways (more on that in a later blog post, mutable method calls are really interesting as they are always like lambdas in other languages).

To optimize code you can get away from mutable and lose this feature:

constant squareArea = length * length

Now there is no recalculation happening and it always will stay its initial value.

January 2023 Multi Line Text Expressions and Introducing New Lines in Strict

January 18, 2023

Murali Tandabany

We added support for multi-line text expressions in Strict following the same approach we used to support multi-line List expressions. Along the same line, we also added support for New Lines inside text expressions. In this blog, we will discuss these both topics in detail.

Multi Line Text Expression

When you want to create a text expression with length more than 100 characters, it is very much possible that you are going to violate the Strict line length limitation which is any line with more than 120 characters in it are not allowed and the compiler will throw an error immediately. To resolve this problem, we are now introducing support for multi line text expressions for the text values with length more than 100 characters at least. If your multi-line text expression character length is below 100, then compiler will not parse this as multi line and throw a compile time error to use single text expression instead.

Creating a multi line text expression is very easy in strict. All you need to do is add a '+' operator at the end of the expression and continue the remaining part of the expression in the next line. Please be aware that you still need to end the text expression before '+' operator by using double quotes and also start the next line with the same tab level and a opening double quotes before continuing the remaining text expression values.

Example:

has log
Run
    constant result = "This is considered to be some interesting text data that has more than hundred character length" +
    "so it is continuing in the next line using strict multi line expression syntax"

It is recommended to use large text content stored in a text file and then load it into the strict program using File type instead of storing them in the program directly.

The below usage of multi-line text expression is not allowed as the line length is below 100 characters.

has log
Run
    constant result = "Has less than hundred character" +
    "so it is not allowed to get compiled"

New Lines in Text

Support for new lines is unavoidable in any programming language. Now, we have added support for New Lines in Strict Text values and in this section, we will discuss the syntax and usages of New Lines in Strict language.

The syntax to insert New Line is as follows,

Text.NewLine

This new line syntax can be used along with the text expression to insert a new line wherever required.

Examples:

has log
Run
    constant result = "First Line - Text data that has more than hundred character length" +
    Text.NewLine +
    "Third Line - so it is continuing in the second next line using strict multi line expression syntax" + Text.NewLine

January 2023 Namings in Strict

January 10, 2023

Murali Tandabany

From the beginning, we never allowed numbers or special characters in the names of anything in Strict. Now, we have decided to allow them with strict limitations for package and type names as explained below. Other than package and type names, all other units in Strict are not allowed to have numbers or any special characters in their names.

Naming Rules for Packages

Package names must start with alphabets only but can contain numbers or '-' in middle or end, all other characters are not allowed.

Examples: Allowed Package Names

Hello-World
Math-Algebra
MyPackage2022

Examples: Not Allowed Package Names

1MathVector
Hello_World
(CustomPackage)

Naming Rules for Types

Type names must start with alphabets only and can contain number in the range from 2 to 9 as the last character. No multiple number digits are allowed in the type name. Another constraint in the type name is name without the last number character should not create any conflict with the existing type name.

Example: Allowed Type Names

Vector2
Matrix3
CustomType9

Example: Not Allowed Type Names

Vector0
Martix1
2Vector
Matrix33
Custom-Type9

Special rule for type names is when you already have an existing type named Text and if you try to created Text2 then it is not allowed because Text2 without 2 is creating a conflict in types therefore it is not allowed.

All other Names

Except package and type, all other names such as method names, variable names, member names and parameter names must have alphabets only and no special characters or numbers allowed.

December 2022 Traits and Components

December 30, 2022

Murali Tandabany

In this blog, we are going to discuss about how we can use a strict type as a trait or as a component in any other type. We will also discuss the differences between these two usages.

After we removed the implement keyword and overloaded that behavior to has keyword, we need a new way to identify whether the type used as a member is intended as a trait or a component at the parser level. For example, see the below code snippet.

Calculator.strict

has App
has file
...

Type Calculator has two members App and file and both types are traits(no implementation provided for the methods). Out of these two, we cannot infer that App type is used as a trait and this program type should implement the Run method from the App type. Also, file type is a component to this type and all the public methods and members of the file type will be used in this program.

Trait

In order to differentiate these two type usages, we have come up with a new rule and this also helps the parser to treat the types as per the intended usage. The new rule is if all of the member type methods are implemented in the current type, then it means that member type is used as a trait.

Example program for member used as a Trait

ConsoleApp.strict

has App
has log
Run
    log.Write(6)

In this example, Run method of the type App member is implemented in the ConsoleApp.strict. With this, compiler now can automatically interpret App member is used as a trait.

Component

On the other side, if no methods of the member are implemented in the current type, then it is automatically interpreted that member type will be a component to this current type and all of it's methods are available for the usage. One more syntax that denotes that the type is used as a component is initializing the member with the constructor parameters. In these cases, compiler will allow the program to get parsed without errors and let the runtime figure out which instance type should be injected as a dependency for this component type member.

Example program for member used as a Component

TextContentPublisher.strict

has File = "test.txt"
ReadAndPublish Text
    Publish(File.Read())

In this example, File member methods are not implemented rather Read method is used inside ReadAndPublish method. Therefore, it is clear now that File member is used as a component in TextContentPublisher type.

Usage Not Allowed

One case which is not allowed in this rule is partially implementing the member type methods. This will create confusion in deciding the member usage, hence it is not allowed and the parser will throw an exception to ask the user to implement missing methods of the member type. The below example program demonstrates this invalid case.

CsvFile.strict

has File
Read Text
    constant openedFile = FileStream.OpenFile
    return openedFile

In this example, CsvFile type implemented Read method but missed to implement Lenght, Write and Delete methods. Therefore, this is not allowed and the compiler will throw an exception in this case.

November 2022 Multi Line List Expressions

November 9, 2022

Murali Tandabany

In strict, majority of the expressions will contain less characters and the length of each line can be contained within 120 character which is the hard limit for any line in strict. One exception is the list type containing more number of elements and in few cases the line length can extend beyond 120 limit. Below is one example program where the list expressions line length goes beyond the maximum limit.

has numbers,
has anotherNumbers Numbers,
GetMixedNumbers Numbers
    (numbers(0), numbers(1), numbers(2), numbers(3), numbers(4), numbers(5), numbers(6), anotherNumbers(0), anotherNumbers(1), anotherNumbers(2))

Multi Line List Expressions

In order to resolve this issue, we have introduced a new feature in Strict to support multi line List expressions. If any list expression which has length above 100 characters, then it is allowed to use multiple lines for the list elements with each list element in a separate line ending with comma. These lines should follow the same indentation as beginning line of the list expression.

The above program should be written as shown below using multi line list expressions.

has numbers,
has anotherNumbers Numbers,
GetMixedNumbers Numbers
    (numbers(0),
    numbers(1)
    numbers(2)
    numbers(3)
    numbers(4)
    numbers(5)
    numbers(6)
    anotherNumbers(0)
    anotherNumbers(1)
    anotherNumbers(2))

Usage Not Allowed

If the total length of the list expression is below 100, then it is not allowed to use multi lines to write those list expressions and should be always written in a single line.

For example below program is not allowed in strict as the total length of the list expression is below 100 characters.

has log
Run
    log.Write((1,
    2,
    3,
    4,
    5,
    6,
    7))

For more information, please refer to the strict example programs in the GitHub repositories.

November 2022 Strict Base and Example folders moved to separate repository

November 2, 2022

Murali Tandabany

In strict, we usually store all the Strict.Base and Strict.Example folder inside the same Strict repository and load it from the path everytime the project starts.

Now, we have moved both Strict.Base and Strict.Examples folder out of Strict source code repository and created a new repository inside strict-lang organization folder. We have also modified the Repository.cs to look for the Strict Base and Example folder files in the development folder (C:/code/GitHub/strict-lang/Strict.Base) and load them. If the files are not available in the development folder, then it will be downloaded from the GitHub repository directly and use them.

One more change is also planned to implement package manager service to automatically sync the latest changes from GitHub to locally cached Strict Base and example files.

Here is the repository link of Strict Base and Example folder files.

Base - https://github.com/strict-lang/Strict.Base

Examples - https://github.com/strict-lang/Strict.Examples

October 2022 Shunting Yard

October 15, 2022

Murali Tandabany

When I was struggling with parsing the nested group expressions, Benjamin Nitschke introduced an algorithm to me called Shuting Yard which solves group expression parsing in a much simpler way. In this blog, we are going to discuss the details about this Shunting Yard algorithm and learn it's usages in Strict.

What is Shunting Yard?

It is a stack-based algorithm which takes arithmetic or logical expressions as input and convert them into postfix notation. The output postfix notation gives us the order of execution of the arithmetic or logical expression operations in order to arrive at the correct result.

More details about the algorithm can be found in this wikipedia page link

https://en.wikipedia.org/wiki/Shunting_yard_algorithm

How Shunting Yard is used in Strict?

We took the complete advantage of Shunting Yard algorithm in the Parser section where each line of code needs to be parsed and spit out into correct expressions. Here, the strict program lines can contain any type of expression such as Number, Text or complex nested group Binary expression with member calls and method calls. In order to parse all theses types of expressions, mainly nested group expressions in the parser without having many performance intensive checks and conditions, we need a simple and clever algorithm like Shunting Yard.

The method expression parser itself takes care of all of the easy kind expressions in the front level which helps to improve the performance of parsing phase. When it comes to complex type expressions, we need to give it as input to the Shunting yard where the complex expression is first tokenized using Phrase Tokenizer class and then each token is ordered based on the operator precendence which then finally gives us back the postfix tokens.

These postfix tokens are stored in the Stack and the method expression parser will pop out each token and construct the result expression with few checks. Specifically, constructing binary expressions out of postfix tokens makes the job very much easier with the use of recursion.

For example:

(2 + (3 + 5) * 5) * 2  converted to -> 2, 3, 5, +, 5, *, +, 2, *

What is Phrase Tokenizer?

Phrase Tokenizer is a class inside Shunting Yard and it is used to tokenize the complex expressions into separate tokens based on certains conditions.

For example:

1 + 2 -> tokenized into "1" , "+", "2"
"Hello + World" + 5 -> tokenized into ""Hello + World"", "+", "5"

For certains types of expression where tokenizing is not helpful, Phrase Tokenizer will spit out the whole expression as single token and then the method expression parser will handle it by splitting the expression into much simpler form and uses Shunting yard again if needed. Mostly, if the expression is nested with arithmetic or logical operators, then tokenizing them will make more sense than for method call or member call expressions.

For example:

Add(4 * 5 + 3 - 1)

This whole expression will be treated as single token in the Phrase Tokenizer and then the method expression parser will split this into separate expressions as shown below

Add as Method call 
4 * 5 + 3 - 1 to Shunting Yard and get output as 4, 5, *, 3, +, 1, -

For more details about Shunting yard and phrase tokenizer, refer to the Strict code base in github.

September 2022 For loops

September 29, 2022

Luka Minjoraia

For loops exist in almost all programming languages (except functional ones) as it serves incredibly useful and convinient practicality for certain problems encountered in day-to-day programming.

Strict works with various forms of for loops and they are very simple to write. The expression in the for loop basically is an iterator which iterates through the series of values. Let's look at a very simple example of the for loop:

for number in numbers
    log.Write(number)

Above is given a very simple example that is pretty straightforward : it iterates through the list of numbers and logs each of the number. numbers is an iterable and number is an explicit iterator variable.

How implicit variables work

We can rewrite previous example in a more convinient, "Strict" way. for loops in Strict has two variables by default - index & value. The aformentioned variables are implicit, meaning they don't need to be declared as they always exist within the scope of the loop.

Let's rewrite the previous example using the implicit value variable:

for numbers
    log.Write(value)

This code does exactly the same thing as above. Note that value is basically a pointer to the instance of the current class (like this in C/C++/C#), but as soon as the scope of the loop is entered, it becomes the iterator variable.

index works similarly, but obviously it holds the index of the element.

for ("Hello", "World", "!")
    log.Write(index)

This would log 0, 1 and 2.

Let's check out this function, which is a part of Range.strict and just sums the value from start of the range to the endo of the range:

Sum
    Range(2, 4).Sum is 2 + 3 + 4
    Range(42, 45).Sum is 42 + 43 + 44
    let result = Mutable(0)
    for item in Range(value.Start, value.End)
        result = result + item
    result

We declare a mutable variable, which is supposed to hold the result of the summation, then we iterate through the Range, perform the sum of the item and return the result. Now this can be rewritten in a much easier way by utilizing the features of Strict.

Sum
    Range(2, 4).Sum is 2 + 3 + 4
    Range(42, 45).Sum is 42 + 43 + 44
    for value
        + value

This does exactly the same as above, the return statement AND the summation is simply inlined in the for scope producing the result.

Nested loops

In Strict, you can take full adventage of nested loops, and they're pretty straightforward.

let weeks = 3
let days = 7
for weeks
    log.Write("Week: " + index + 1);
    for days
        log.Write("Day: " + index + 1)

September 2022 Details about Mutability in Strict

September 20, 2022

Murali Tandabany

In Strict programming language, every member or variable is immutable by default. i.e. value can be only assigned during member/variable initialization. Thus, any has or let is constant by default. Strict also supports Mutable types so that member/variable values can be changed after initialization as well. This can be achieved by explicitly specifying the type as Mutable during initialization or if the type implements Mutable trait.

How to define Mutable types?

There are three ways to define Mutable types. 1. Explicitly mention the type as Mutable during member/variable assignment.

```
has counter = Mutable(0)
let counter = Mutable(0) //now counter variable is mutable and its value can be changed
for Range(0, 5)
    counter = counter + 1
```

2. Use the types that implements Mutable as your member/variable type.

```
has counter = Count(0) //here Count is the type that implements Mutable in their class implementation
let counter = Count(0)
```

3. Directly use the type which implements Mutable trait in their class

```
has count //here count is the type that implements Mutable 
```

All of the above three ways does the same operation and enable mutability to member/variable in strict.

Immutable Types Cannot Be Reassigned

Parser always checks the type of the member/variable before reassigning value to it. When any immutable type values are reassigned in strict program, parser will throw ImmutableTypesCannotBeReassigned exception.

has number 
Run
  number = number + 5 -> this will cause ImmutableTypesCannotBeReassigned error, so we MUST use Mutable types when we want to change values

July 2022 Introduction To Generics In Strict

July 1, 2022

Benjamin Nitschke & Murali Tandabany

It is true that we can have programming languages without supporting generics like first versions of Java (till much later) or C# (till .net 2.0) or Go (for 12+years). They didn't have generics at all and a programmer could do everything still fine using polymorphism and runtime checks as well. However, generics make things not just more flexible, but allow the compiler to do MANY checks before anything even runs. You can think of it as "anything can be a object" casted down to specific interfaces.

How did we start implementing Generics in Strict?

We had to disallow ANY "any" for the moment to make the focus on generics very clear, we don't want the "Any" Type anymore except for the automatic base class (ala "object" in c#). This made many things to not compile or work. These things didn't really work anyway, they just compiled and would not run (except if just using "object" in the transpiled c#, but in cuda there is no equivalent)

How did we fix it?

The first thing that didn't work anymore was all of the methods returning Any because we didn't had generics or conversions to get a type of an instance or converting from one instance type to another, e.g. BinaryOperator.strict (while this is not used directly, it directs on what the compiler will parse it as)

to(type) returns Any
to(any) returns Any

And there are many classes that use unknown things like HashCode.strict (which also will never be used directly, it is just explaining how it works)

has any -> these are clearly forbidden now, so we MUST use generics here or have a way to create types in code

to(type) returns Type(type)

The above line of code makes just more sense, the type will be the generic Type, we probably could just say "type" and mean "Type(type)" by that, which is a mouthful

to(any) // this is simply forbidden, you can only have specific types with to operator
to(Type(Number)) returns Number or
to(Type(Text)) returns Text

For HashCode just remove "has any", everything is already any, so no need to include it, the only problem here is when using "this" or "value" inside a method or expression, it will not be clear from where those are coming, but again this class is NOT used directly, it is just to explain how HashCode are calculated:

implement Number
Compute returns Number
    if value
        is Number
            return value
        is Text
            return value.Length
    else
        return 0

(btw: I invented switch statements here by allowing to split if expression into multiple choices), the "if value" can also be optimized to "if", which just forwards the "value" from the above scope to below. But this is yet to be implemented in the near future

How to use Generics in Strict program?

Input.strict base type should be limited to what works for now and what is needed, which is Text reading

Read returns Text

If we need numbers or bytes or parsing in the future (xml, json, yaml, csv, etc.), we will add it when needed like ReadNumbers(separators Texts) returns Numbers, but we probably should think about how bytes make it through, how we can avoid saying "ReadNumbers" and then also having to say "returns Numbers", too much fluff

Output.strict is more complicated and could be solved with generics, but that is a really hard example to start with, also normally you just have a ton of methods for each type you allow to write (e.g. in java, c#, go, std, etc.), instead we just add the methods we need:

Write(text)
Write(number)

Remaining methods like Write(xml), Write(json), Write(type), etc. can be added when needed and we are not that far yet. Currently there is no method overloading, so maybe this is internally just a generic implementation or we just allow different parameters for the same named method (and just point to the correct one as we know each type at compile time so far). Only problem here is if there is polymorphism (which we also don't have yet), then the decision has to be still done at run time (compiler can only check that it makes sense up to the trait/interface)

Applications of Generics

The first and important application of Generics inside Strict language is List which is the reason for implementing Generics at this time. Therefore, after Generics, statements like

has anys -> won't be supported

Instead, we need to mention the specific type for example,

has numbers -> List of Number
has elements -> Numbers indirectly mean the same List of Number type

This means numbers is a type of List of Number and in this way the compiler will directly know the type without the need of conditional castings and thus makes all the operations of the type faster than before.