• Background Image

    MSL Specifications

    January 21, 2020

January 21, 2020

MSL Specifications

The following is a brief excerpt from the MSL Language Specifications 1.0, a 70-page comprehensive introduction to a new programming language for applications built with Mimix technology. I hope this tiny introduction encourages you to download and read the full MSL Spec, and to email me with your comments, questions, and ideas! — D

 

Mimix Stream Language is a domain-specific language designed for recording and playing back text editing sessions. A list of MSL expressions is an indelible record of source materials, original writing, editing, and notes in the order in which they were introduced. MSL is the filament in the Mimix light bulb. It is the most important part of the system.

 

Introduction

MSL provides high level operations for creating and destroying atoms, building and applying metadata, recording and sharing streams, and accessing and editing these elements.”
— Mimix Whitepaper

Beyond Word Processing

A word processor knows about words, sentences, paragraphs, pages, and documents. It does not, however, understand the factual information contained in what you read and write. Mimix represents the evolution of the word processor in that it provides a means for text processing software to interpret, display, analyze, compare, correct, and transform the factual data inside our documents.

The Mimix whitepaper describes in detail the historical precedents for a different kind of reading and writing system. It also lays out several scenarios in which such a system can radically transform how we interact with written documents. In short, this demands several new capabilities from our software:

  1. Fact checking and correction of factual errors in written text.
  2. Options for viewing facts in multiple formats.
  3. Access to source materials for all references.
  4. A “rewind” facility to return to any previous version of text.
  5. A portable version that can be shared with all references intact.

Mimix is a new way of looking at research and writing. To implement it requires new tools and a new vocabulary. When this document first mentions a word that has a special meaning in Mimix, it will appear in bold.

Why Create a Language?

The Mimix ecosystem is built around a new programming language called MSL, Mimix Stream Language. This document describes that language in detail. A new programming language was desirable for several reasons:

  1. It allows modeling an editing session as a program which can be audited and replayed.
  2. It is easier to read and write than the underlying system code.
  3. It provides built-in protections that the underlying system cannot.
  4. It serves as a common specification between all the parts of the system.
  5. It enables anyone to write software that works with Mimix.

Recording a series of text documents and edits in the form of a program allows thinking about an editing session in programming terms. We can view words and paragraphs as variables and values. Text references such as a document we’re quoting can be viewed as dependencies. This lets us extract any part of the program (the editing session), find its text dependencies, and work with them using any software that understands MSL.

The MSL language is designed to be easier to write than the actual system code which is needed to perform the same functions. You can think of MSL as a shortcut method for describing reading and writing as they take place in linear time.

In most cases, Mimix users will never see MSL, but it’s designed to easy to read by human beings regardless. This makes testing, debugging, and extending the system easier.

The MSL language provides built-in protections for data that the underlying system would not. This is used to enforce protections for factually correct information, which we call canon. MSL also uses a built-in set of rules to prevent overwriting important data. Most importantly, it offers a granular rewind capability in which any piece of data can be “rewound” to any of its previous values. This same ability allows extracting any value in the system for use in a different context with all of its dependencies and sources intact. These features are unprecedented among programming languages and are defining aspects of MSL.

The MSL language is communicated between a viewer, where text editing takes place, and an MSL engine which records the expressions in a hybrid database which offers access to both past and present values. This allows the development of numerous compatible engines and viewers, all of which rely on the present MSL specification.

Anyone can write tools that work with MSL and this is encouraged by specifying a language definition that is simple, complete, and coherent. Developers can easily visualize the MSL they will need to input or output in order to achieve their desired results.

Language Purpose

Mimix Stream Language is a domain-specific language designed for recording and playing back text editing sessions. A list of MSL expressions is an indelible record of source materials, original writing, editing, and notes in the order in which they were introduced. MSL is the filament in the Mimix light bulb. It is the most important part of the system.

MSL is designed to be good at two kinds of jobs. First, it can be used to record the actions of a Mimix user as he or she reads, writes, edits, and annotates text. It provides good traceability in that the source of values and their changes over time can be readily ascertained. It enables a rewind facility so that previous versions of a stream can be viewed, extracted, or rolled back.

Second, it can be used as a playback language to recreate the same view or a different view of the recording. This allows recreating an entire view or stream from the MSL text at any point, or creating a new type of presentation or transformation of the materials a user has viewed, written, edited, or annotated (or any subset of these) up to a certain point. In other words, MSL can be used to play back any part of the stream with a different view than the original.

Language Implementation

MSL is a stream recording language which captures a user’s reading, annotations, writing, and editing into a stream of views as they take place in the viewer. User behaviors are recorded as a series of MSL expressions. These can be used individually, in parts, or in a sequence to recreate any part of the stream using the same view, a different view, or a different viewer.

MSL expressions are executed by a state machine. The state is stored in a hashtable in the MSL engine. MSL expressions passed to the engine can get or set values in the hashtable. The current value of the hashtable represents the full system state after the last MSL expression executed. The state machine serves as a convenient reference to the values contained in the stream, which are also preserved separately.

MSL is an imperative language in which every expression recalls or updates the system state in a sequential order. The state is defined as the value of the hashtable up to and including the execution of the last MSL expression. Although MSL is not procedural, it does allow the application of selectors and transforms, functions which are applied to an atom, the basic unit of data in the language. Selectors and transforms permit viewers and engines to define functions outside of MSL which are then acted upon by MSL values.

MSL is lexically scoped with atoms having values defined in the atoms themselves, their metadata, in their applied canon, or in one of these belonging to aprevious containing view, stream, world, or machine.

MSL is a functional language in which every program contains all its literal values inline and all functions can be resolved through variable substitution without reliance on an independent state. MSL is also functional in the style of the Lambda Calculus in that every MSL expression is permitted only one parameter, its value. MSL supports currying values in that the result of single-argument functions are passed upwards to other functions in order to provide a final value.

The hybrid database in a Mimix system serves only as a convenience for quick resolution of an atom’s value. The actual values themselves are always recorded in MSL text and can be resolved by simply examining the MSL text backwards without reference to the hybrid database.

MSL is not Turing complete and essentially consists of only elaborate variable getters and setters. No MSL expressions perform looping, testing, branching, or recursion.

MSL is an interpreted language with the system state stored in a nested series of hashtables backed up by a stream of MSL text. MSL expressions which set an atom’s value change the system state by updating values in the hashtables and recording a new entry in the text. MSL expressions which only return a value do not change the system state and simply recall the current value from the hashtables or the text while also recording the expression itself in MSL text.

Beginning MSL

The simplest way to provide a permanent record of what you read and write is to write down everything you do in chronological order. In fact, this is how MSL works.

If you were planning a trip to Hawaii, you might read two books and make some notes:

  1. (@fodor-book Fodor’s 2020 Guide to Honolulu and Oahu)
  2. (@lonely-planet Lonely Planet Hawaii)
  3. (@hawaii-notes International Marketplace. Pearl Harbor. Shave ice.)

These are MSL expressions. Whenever a valid MSL expression appears in this document, we’ll use a code font in blue. When MSL concepts are referenced without a full, valid syntax, we’ll use a code font in black.

The (@) notation here refers to an atom, the basic unit of storage in MSL. Every atom includes a key which identifies it. It’s customary (but not required) to write the key up against the @ sign.

(@lonely-planet Lonely Planet Hawaii)

This atom contains the title for a book. In order to be useful as a reference going forward, we would need to record the entire book. We could simply dump the book’s contents into the atom. Or we could give individual paragraphs, people, places, etc. their own atoms that sit inside the book atom.

(@lonely-planet Lonely Planet (@Hawaii)

   (@intro The goddess (@Pele) …)

   (@chapter1 (@p1 When the ancient (@Hawaii Hawaiians)…) (@p2 …))

   (@chapter2 …)))

Text which is read, written, or edited by a Mimix user is recorded sequentially in a series of atoms, and this will be shown through many examples.

Atoms can be grouped into views, streams, worlds, and machines. Atoms can also be defined in canon, giving them a type of data protection. Views, streams, worlds, machines, and canon are atoms that reside in their own namespaces. These basic concepts enable all five of the abilities we need to go beyond word processing.

» Continue reading the full specifications…

— D 1/21/2020