how it works

How Pangine works

This page follows a question through the engine: how values are stored, how experience is remembered, how a question finds its answers, how evidence becomes probability, and how graded questions and partitions extend the same steps.

The examples teach the language, and the grammar page is its reference. The transcripts here are literal output from the current console: text after command> is the input, and indented text is the output. inspect is a console command that lists an answer's values with their evidence counts, probabilities, and sources. In outline, a question passes through these steps:

subject + question
    -> structural matches against complete sources
    -> blanks joined where the question repeats them
    -> complete rows, each with the sources that prove it
    -> one shared answer behind the question's output Percepts
    -> read with $, reveal with &, choose with ^, or draw with ^~

One kind of value

Everything Pangine stores, asks, or answers is a Concept, and there are four kinds. A named Concept such as [cat] is an opaque name. An ordered Concept such as [cat]->[purrs] is a sequence whose positions matter. An unordered Concept such as [cat][dog] is a collection whose members each carry a signed whole-number coefficient. A Percept such as {memory} is a named reference that holds a value. The console prints the members of an unordered value on separate lines.

command> [B][A][A]
  x2 [A]
  [B]
command> ([A][B])([A][B])
  x2([A][B])
command> ([A][B])*([A][B])
  x2 [A]
  x2 [B]
command> [A]![A]
  []

Every Concept has one canonical form. An unordered Concept keeps its members in a fixed order and adds the coefficients of equal members, so [B][A][A] becomes x2[A][B]. Parentheses keep a complete member, while * merges the members of its operands. Inversion is a coefficient of -1, so [A]![A] cancels to no Concept. Coefficients are structure: x2 does not by itself mean two events.

Each engine interns its Concepts. Building one looks it up by a fingerprint of its structure, and complete structural equality decides whether it already exists, so equal Concepts are one object and questions, answers, and remembered experience share structure instead of copying it. The engine holds its registry through weak references, so a Concept that nothing uses any longer is freed.

Memory and experience

A Percept holds either a current value or remembered experience. = replaces a current value. ~= remembers one complete experience: the Percept keeps each distinct experience with an evidence count, a signed 64-bit integer, and repeating an experience adds one to its count. The Percept's value is the union of its experiences with their counts as coefficients, which is why a repeated experience prints with x2.

command> {world} ~= [morning]->[birds]
  [morning]->[birds]
command> {world} ~= [morning]->[birds]
  x2([morning]->[birds])
command> {world} ~= [morning]->[traffic]
  x2([morning]->[birds])
  [morning]->[traffic]

When ~= remembers an experience that mentions input Percepts holding current values, it captures those values at that moment, so later changes do not rewrite old experience. Recording updates the remembered value instead of rebuilding it: 20,000 tic-tac-toe board experiences, 2,189 of them distinct, record in about 67 ms in a native release build. The read-only global Percept {*} supplies the ordinary Concepts currently live in the engine.

Asking a question

A question is a Concept whose Percepts are blanks. subject @ question matches it against each complete source the subject supplies: each experience a selected Percept remembers, or a structural subject itself. Matching is structural and recursive. A question can match a whole source or any part inside it, and a flat ordered question can match a contiguous part of a longer ordered Concept. A match binds each blank to a Concept.

Each match stays tied to the complete source it came from. Parts of a question that share no blank must be completed by one source, so two remembered experiences cannot be mixed into an answer that neither supports. A blank that appears in two parts must take the same value in both, which is how a question joins relationships, within one source or across several. While matching, the engine also records where in a source each part matched, so equal Concepts at different positions are not confused. Sections 5 to 7 of the examples show each of these rules.

Before matching, the engine narrows the sources through an index of each Percept's experiences: the shapes they contain, the names they mention, and the name at each position of their ordered Concepts. The index is built at a Percept's first question, kept current as experience arrives, and dropped when the Percept is replaced. An exact question over 20,000 remembered tic-tac-toe boards visits only the boards with the same cells and takes about 0.09 ms natively once the index exists.

Answers

A question returns complete rows. Each row is the whole question with its blanks filled, together with the sources that prove it. Instead of reducing the rows to separate lists, Pangine keeps them in one answer that every output Percept of the question links to. The answer is itself an ordinary Concept holding the active outputs, the rows, each row's sources and counts, the current projections, and a revision. Because it is a Concept, it can be formatted and parsed into another engine, and its question shape can itself be questioned. Continuing in the same console:

command> {world} @ [morning]->{sound}
  [morning]->[birds]
  [morning]->[traffic]
command> &{sound}
  [morning]->{sound}
command> inspect {sound}
  * +2, p=2/3, 1 row: [birds]
      +2 from {world}: [morning]->[birds]
    +1, p=1/3, 1 row: [traffic]
      +1 from {world}: [morning]->[traffic]
command> ^{sound}
  [birds]

& reveals the question behind an output, and $ reads an output's current projection. ^ chooses the most probable value of one output, keeps only the rows compatible with it, and recalculates every linked output, so outputs chosen one at a time stay consistent. A later question can reuse linked outputs as blanks and extend the same answer. @+= and @-= add or subtract the evidence of matching rows from another answer, which is how the tic-tac-toe demo applies wins and losses. Publishing a new answer to linked outputs requires every one of them to still hold the answer it was derived from, so a stale write is rejected.

Evidence and probability

An answer reads evidence counts as probabilities. A row proven by one experience weighs that experience's count. A row joined from separate experiences weighs the product of their counts, while one experience that proves several parts of a row counts once. A value's evidence is the sum over the rows that project it, and its probability is its share of the positive evidence among the alternatives. A value whose evidence is zero or negative has probability zero.

command> {knowledge} ~= [Socrates]->[is-a]->[human]
  [Socrates]->[is-a]->[human]
command> {knowledge} ~= [Socrates]->[is-a]->[human]
  x2([Socrates]->[is-a]->[human])
command> {knowledge} ~= [human]->[is-a]->[mortal]
  x2([Socrates]->[is-a]->[human])
  [human]->[is-a]->[mortal]
command> {knowledge} @ ([Socrates]->[is-a]->{kind})({kind}->[is-a]->{conclusion})
  [Socrates]->[is-a]->[human]
  [human]->[is-a]->[mortal]
command> inspect {conclusion}
  * +2, p=1, 1 row: [mortal]
      +2 from 2 sources:
        {knowledge}: x2([Socrates]->[is-a]->[human])
        {knowledge}: [human]->[is-a]->[mortal]

Here [Socrates]->[is-a]->[human] was remembered twice and [human]->[is-a]->[mortal] once, so the joined row weighs 2. ^ takes the value with the greatest positive count, with canonical order breaking a tie, and ^~ draws a value with probability equal to its share.

The arithmetic is integer throughout. Probabilities stay exact fractions while their arithmetic fits in 128 bits, and past that they continue in fixed point with 64 fractional bits. Each engine draws from its own seeded generator, which starts at seed 0, and lays out the candidates in canonical order, so the console, the Rust API, and this site's browser runtime compute the same probabilities and draw the same values.

This reading descends from Pangine's first version, which kept a probability and a strength for each piece of evidence and chose the greatest probability times strength. A count is that product, and adding whole counts makes the merge exact.

Graded questions

@ answers only with rows that remembered experience proves whole. @~ asks the same question at wider levels and labels every row with the level it came from:

Exact rows are the rows @ returns.
Composed rows let parts of the question that share no blank come from separate experiences.
Generalized rows come from comparing the question with each remembered experience as one complete case whose names may differ, each at its own distance.

command> {closet} ~= ([top]->[red])([bottom]->[jeans])
  [bottom]->[jeans]
  [top]->[red]
command> {closet} ~= ([top]->[blue])([bottom]->[skirt])
  ([bottom]->[jeans])([top]->[red])
  ([bottom]->[skirt])([top]->[blue])
command> {closet} ~= [top]->[green]
  [top]->[green]
  ([bottom]->[jeans])([top]->[red])
  ([bottom]->[skirt])([top]->[blue])
command> {closet} @~ ([top]->{shirt})([bottom]->{pants})
  ([bottom]->[jeans])([top]->[blue])
  ([bottom]->[jeans])([top]->[green])
  ([bottom]->[jeans])([top]->[red])
  ([bottom]->[skirt])([top]->[blue])
  ([bottom]->[skirt])([top]->[green])
  ([bottom]->[skirt])([top]->[red])
command> inspect {shirt}->{pants}
  * +1, p=1/3, 1 row: [blue]->[skirt]
      +1 from {closet}: ([bottom]->[skirt])([top]->[blue])
  * +1, p=1/3, 1 row: [red]->[jeans]
      +1 from {closet}: ([bottom]->[jeans])([top]->[red])
    +1, p=1/12, 1 row: [blue]->[jeans]
      +1 composed from 2 sources:
        {closet}: ([bottom]->[jeans])([top]->[red])
        {closet}: ([bottom]->[skirt])([top]->[blue])
    +1, p=1/12, 1 row: [green]->[jeans]
      +1 composed from 2 sources:
        {closet}: ([bottom]->[jeans])([top]->[red])
        {closet}: [top]->[green]
    +1, p=1/12, 1 row: [green]->[skirt]
      +1 composed from 2 sources:
        {closet}: ([bottom]->[skirt])([top]->[blue])
        {closet}: [top]->[green]
    +1, p=1/12, 1 row: [red]->[skirt]
      +1 composed from 2 sources:
        {closet}: ([bottom]->[jeans])([top]->[red])
        {closet}: ([bottom]->[skirt])([top]->[blue])

Two outfits were remembered whole, and the green top alone. @ would return only the two whole outfits. @~ also composes four pairs never seen together. To score them, each level is read against the next wider one with the Witten-Bell rule, which comes from text compression: P(v) = (c(v) + T * P_next(v)) / (c + T), where c(v) is the value's count at the level, c the level's positive total, and T the number of different rows the level supports. The composed level supports six pairs with one count each, so each reads 1/6 there. The exact level supports two pairs with a total of 2, so red with jeans reads (1 + 2 * 1/6) / 4 = 1/3, and green with jeans reads (0 + 2 * 1/6) / 4 = 1/12. Whole observations keep priority, and the rule needs no tuned parameter. The interpolation runs once over an answer's complete rows, and reading one output adds the rows that give each of its values, so it always agrees with reading them all.

command> {families} ~= ([Tom]->[parent-of]->[Bob])([Bob]->[parent-of]->[Ann])([Tom]->[grandparent-of]->[Ann])
  [Bob]->[parent-of]->[Ann]
  [Tom]->[grandparent-of]->[Ann]
  [Tom]->[parent-of]->[Bob]
command> {families} ~= ([Liz]->[parent-of]->[Max])([Max]->[parent-of]->[Ivy])([Liz]->[grandparent-of]->[Ivy])
  ([Bob]->[parent-of]->[Ann])([Tom]->[grandparent-of]->[Ann])([Tom]->[parent-of]->[Bob])
  ([Liz]->[grandparent-of]->[Ivy])([Liz]->[parent-of]->[Max])([Max]->[parent-of]->[Ivy])
command> {families} @~ ([Joe]->[parent-of]->[Sue])([Sue]->[parent-of]->[Kim])([Joe]->[grandparent-of]->{who})
  x2(([Joe]->[grandparent-of]->[Kim])([Joe]->[parent-of]->[Sue])([Sue]->[parent-of]->[Kim]))
command> inspect {who}
  * +2, p=1, 2 rows: [Kim]
      +1 at distance 5 from {families}: ([Bob]->[parent-of]->[Ann])([Tom]->[grandparent-of]->[Ann])([Tom]->[parent-of]->[Bob])
      +1 at distance 5 from {families}: ([Liz]->[grandparent-of]->[Ivy])([Liz]->[parent-of]->[Max])([Max]->[parent-of]->[Ivy])

The generalized level compares the question with one remembered experience at a time, with the question's names inside ordered relationships opened into hidden blanks. Each name occurrence that differs from the question costs one, plus one for each extra value that a name the question repeats would take. Joe, Sue, and Kim differ from each family's people in five occurrences, so each family sits at distance 5. An experience counts only if it shares a name with the question and supplies each part of the question from a different part of itself. When every occurrence of a renamed question name sits on the case's answer, the answer maps back to that name, so each family answers [Kim] in the question's own terms. Each distance forms one more level below the composed rows, and the same interpolation reads it.

Partitions

Remembered experience can also be divided among partition engines. Each experience goes to the partition chosen by a hash of its canonical spelling, so its whole count lives in one place. A question goes to every partition. Each matches its own experiences and returns a partial answer: the evidence for each part of the question and, for @~, its generalized rows. The engine the program talks to merges the partial answers, joins and grades the rows, and computes the same answer one engine would. Losing a partition leaves exactly the answer to the experience that remains.

command> {world} ~= [morning]->[birds]
  [morning]->[birds]
command> {world} ~= [morning]->[birds]
  x2([morning]->[birds])
command> {world} ~= [morning]->[traffic]
  x2([morning]->[birds])
  [morning]->[traffic]
command> {world} ~= [evening]->[crickets]
  x2([morning]->[birds])
  [evening]->[crickets]
  [morning]->[traffic]
command> partitions 2
  2 partitions
  partition 0: {world} 2 experiences
  partition 1: {world} 1 experience
command> {world} @ [morning]->{sound}
  [morning]->[birds]
  [morning]->[traffic]
command> inspect {sound}
  * +2, p=2/3, 1 row: [birds]
      +2 from {world}: [morning]->[birds]
    +1, p=1/3, 1 row: [traffic]
      +1 from {world}: [morning]->[traffic]

Partial answers travel between engines as graphs that hold each distinct Concept once. Native builds run each partition on its own thread, and the browser runs them in turn. Dividing pays off when matching dominates: on a 16-core machine, a graded question over 8,000 remembered families took 717 ms in one engine and 375 ms across 8 partitions, while a small question pays for the trip to every partition and back. The fourth demo divides a memory in this browser.

The implementation

Pangine is a Rust library of about 14,000 lines, including its tests and the browser binding, with one dependency, im, for persistent vectors. It uses no unsafe code. Results are deterministic: collections iterate in a fixed order, arithmetic is integer, and sampling is seeded. The console runs commands interactively or from a file, and a WebAssembly binding runs the same crate in the browser for the workbench and the demos on this site, so nothing is sent to a server.

About 250 tests cover the behavior: ordinary tests by topic, seeded property tests, and differential tests. One property test compares indexed and full-scan answers for 1,024 generated questions, and others check that partial answers reduce to one engine's answer in any grouping and arrival order. A browser check drives this site's workbench and demos in a headless browser, and continuous integration runs the suite on Linux, macOS, and Windows.

Limits

A completed row can contain a detail supplied only by the question, so a completed answer is not by itself verified. The Rust API keeps those remainders; the language has no direct form for them yet.
Reading linked outputs directly preserves their rows, but an alias or an expression that mixes linked outputs with ordinary values can lose the pairing.
A match inside a coefficient or an inverted member counts as support at the experience's own count, so remembering !([board]->[bad]) supports [board]->[bad]. Counterevidence goes through @-=.
A broad question can still create substantial matching work. There is no query planner, persistence, or streaming interface.
Formatted text repeats shared structure, so a large answer can be much larger as text than in memory.
Partitions share one process, and Pangine has no network protocol.

Probabilities are relative frequencies of remembered evidence, not calibrated confidence, and names mean nothing beyond the structure and experience a program gives them.