BitBloomTech · mtinning · Dec 12, 2019 · Oct 24, 2019 · Oct 30, 2019 · Oct 30, 2019
diff --git a/docs/user_guide/input-file-definition.md b/docs/user_guide/input-file-definition.md
@@ -4,7 +4,7 @@ Spawn input is a hierarchical structure of branching nodes which allows large nu
 
 ## Getting Started
 
-The specification is defined in an object named `"spec"`. Each name/value pair within this object is a parameter name and its value. The following generates a single specification node with one parameter, named `"alpha"` with a vlue of 4:
+The specification is defined in an object named `"spec"`. Each name/value pair within this object is a parameter name and its value. The following generates a single specification node with one parameter, named `"alpha"` with a value of 4:
 ```json
 {
     "spec": {
@@ -13,7 +13,7 @@ The specification is defined in an object named `"spec"`. Each name/value pair w
 }
 ```
 
-Sibling name/value pairs are simultaneous (i.e. orccur on the same node). The following generates a single node with *two* simulataneous parameters - `"alpha"` with a value of 4, and "beta" with a value of "tadpole":
+Sibling name/value pairs are simultaneous (i.e. occur on the same node). The following generates a single node with *two* simultaneous parameters - `"alpha"` with a value of 4, and "beta" with a value of "tadpole":
 ```json
 {
     "spec": {
@@ -34,7 +34,7 @@ Separate nodes can be created by separating parameters into different JSON nodes
 }
 ```
 
-An identical specifcation could be written (less concisely) as:
+An identical specification could be written (less concisely) as:
 ```json
 {
     "spec": {
@@ -44,7 +44,7 @@ An identical specifcation could be written (less concisely) as:
 }
 ```
 
-Avoiding repetitive definition and enabling concise and readable but compex specifications is one of the key aims of Spawn.
+Avoiding repetitive definition and enabling concise and readable but complex specifications is one of the key aims of Spawn.
 
 ## Arrays
 
@@ -89,7 +89,7 @@ There is no limit to the number of sibling arrays, but they *must* all have equa
 
 ## Value Proxies
 
-The value of parameter/value pairs can be represented by a proxy. The proxy is a string that starts with either a type identifier followed by a colon (longhand) or a special character (shorthand) to determine which type of value proxy it is. The parser then replaces the proxy when the specification is resolved. The tpyes of value proxies are as follows:
+The value of parameter/value pairs can be represented by a proxy. The proxy is a string that starts with either a type identifier followed by a colon (longhand) or a special character (shorthand) to determine which type of value proxy it is. The parser then replaces the proxy when the specification is resolved. The types of value proxies are as follows:
 
 | Type | Longhand | Shorthand | Description |
 |------|----------|-----------|-------------|
@@ -151,7 +151,7 @@ The following example generates a value of 4 for "alpha" via the "a" object and
 
 ### Evaluators
 
-Evaluators allow function-style syntax to evaluate expressions with arguments. Arithmetic operations are supported as well as inbuilt evaluators `range`, which produces an evenly speced array, and `repeat`, which repeats a particular value. Unlike macros and generators, evaluators do not need an object defined alongside the `spec`. Some examples:
+Evaluators allow function-style syntax to evaluate expressions with arguments. Arithmetic operations are supported as well as inbuilt evaluators `range`, which produces an evenly spaced array, and `repeat`, which repeats a particular value. Unlike macros and generators, evaluators do not need an object defined alongside the `spec`. Some examples:
 
 | Example | Resolution |
 |---------|------------|
@@ -163,7 +163,7 @@ Evaluators allow function-style syntax to evaluate expressions with arguments. A
 | `"#range(0.3, 0.5, 0.1)"` | `[0.3, 0.4, 0.5]` |
 | `"eval:repeat(5, 3)"` | `[5, 5, 5]` |
 
-Note that the `repeat` can be used with a generator as argument and therefore generate a different value for each elemtn of the array. Evaluators can also take other parameters simultaneously present in the specification if they are prefixed by `!`. They do not need to be in the same object, but if not they must be defined higher up the object tree (i.e. they are unreferencable if in sub-objects). The following resolves `"gamma"` into the list `[3, 4]`:
+Note that the `repeat` can be used with a generator as argument and therefore generate a different value for each element of the array. Evaluators can also take other parameters simultaneously present in the specification if they are prefixed by `!`. They do not need to be in the same object, but if not they must be defined higher up the object tree (i.e. they are not referenceable if in sub-objects). The following resolves `"gamma"` into the list `[3, 4]`:
 ```json
 {
     "spec": {
@@ -176,7 +176,7 @@ Note that the `repeat` can be used with a generator as argument and therefore ge
 }
 ```
 
-Whenu referencing a parameter in an arithemtic operation, the `#` is no longer needed (but the `!` is required):
+When referencing a parameter in an arithmetic operation, the `#` is no longer needed (but the `!` is required):
 ``` JSON
 {
     "spec": {
@@ -186,9 +186,26 @@ Whenu referencing a parameter in an arithemtic operation, the `#` is no longer n
 }
 ```
 
-### Resolution Order
+## Literals
 
-work in progress
+There are cases in which it is desired that specification value does not take on its default spawn interpretation. For this, there is the concept of a literal. This is done by prefixing the parameter name with `~`. The following will generate a single node where `alpha` is an array (passed through to the spawner as a list) and `beta` is a string starting with `$` (rather than looking up a macro). Arrays, objects, all (apparent) value proxies and equations can be taken as literal and therefore not expanded or looked up:
+``` JSON
+{
+    "spec": {
+        "~alpha": ["egg", "tadpole", "frog"],
+        "~beta": "$NotAMacro"
+    }
+}
+```
+
+Literals can also be specified on the value-side. This can be particularly useful when it is desired to expand literals as part of an expansion. In this case, the value must always be a string, but if the string succeeding the literal prefix is JSON serialisable it will be serialised as such, otherwise the value will remain a string. For example, the following produces three nodes, each with an array as the value of the `alpha` parameter:
+```JSON
+{
+    "spec": {
+        "alpha": ["~[1, 2]", "~[3, 4]", "~[5, 6, 7]"]
+    }
+}
+```
 
 ## Policies
 

diff --git a/spawn/parsers/constants.py b/spawn/parsers/constants.py
@@ -27,6 +27,7 @@
 POLICY = 'policy'
 PATH = 'path'
 GHOST = '_'
+LITERAL = '~'
 
 GENERATOR_SHORT = '@'
 MACRO_SHORT = '$'

diff --git a/spawn/parsers/specification_parser.py b/spawn/parsers/specification_parser.py
@@ -16,7 +16,7 @@
 # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301  USA
 """This module defines the ``SpecificationParser``, which parsers specifications
 """
-from json import load
+import json
 from copy import deepcopy
 
 from spawn.errors import SpecFormatError
@@ -36,7 +36,7 @@
 from .constants import (
     COMBINATOR, ZIP, PRODUCT,
     RANGE, REPEAT, MULTIPLY, DIVIDE, ADD, SUBTRACT,
-    PATH, POLICY, GHOST
+    PATH, POLICY, GHOST, LITERAL
 )
 from .value_libraries import ValueLibraries
 from ..specification import generator_methods
@@ -89,7 +89,7 @@ def get(self):
         :rtype: dict
         """
         with open(self._input_file) as input_fp:
-            return load(input_fp)
+            return json.load(input_fp)
 
 class DictSpecificationProvider(SpecificationDescriptionProvider):
     """Implementation of :class:`SpecificationDescriptionProvider` that reads
@@ -238,19 +238,20 @@ def _merge_policies(left, right):
         return {**left, **right, **merged}
 
     def _parse_value(self, parent, name, value, next_node_spec, node_policies, ghost_parameters):
+        literal, name, value = self._parse_literal(name, value)
         # combinator lookup
         if self._is_combinator(name):
             self._parse_combinator(parent, name, value, next_node_spec, node_policies, ghost_parameters)
         # list expansion
-        elif isinstance(value, list):
+        elif isinstance(value, list) and not literal:
             for val in value:
                 self._parse_value(parent, name, val, next_node_spec, node_policies, ghost_parameters)
         # burrow into object
-        elif isinstance(value, dict):
+        elif isinstance(value, dict) and not literal:
             self.parse(value, parent, node_policies=node_policies, ghost_parameters=ghost_parameters)
             self.parse(next_node_spec, parent, node_policies=node_policies, ghost_parameters=ghost_parameters)
         # rhs prefixed proxies (evaluators and co.) - short form and long form
-        elif isinstance(value, str) and self._is_value_proxy(value):
+        elif isinstance(value, str) and self._is_value_proxy(value) and not literal:
             next_parent = ValueProxyNode(
                 parent, name, self._value_proxy_parser.parse(value),
                 node_policies.get(PATH, None), ghost_parameters
@@ -259,10 +260,20 @@ def _parse_value(self, parent, name, value, next_node_spec, node_policies, ghost
         # simple single value
         else:
             next_parent = self._node_factory.create(
-                parent, name, value, node_policies.get(PATH, None), ghost_parameters
+                parent, name, value, node_policies.get(PATH, None), ghost_parameters, literal=literal
             )
             self.parse(next_node_spec, next_parent)
 
+    def _parse_literal(self, name, value):
+        literal_key = self._is_literal(name)
+        literal_value = self._is_literal(value)
+        literal = literal_key or literal_value
+        if literal_key:
+            name = self._deliteral(name)
+        elif literal_value:
+            value = self._deliteral(value)
+        return literal, name, value
+
     def _is_value_proxy(self, value):
         return self._value_proxy_parser.is_value_proxy(value)
 
@@ -287,8 +298,10 @@ def _get_next_node(self, node_spec):
         next_key = list(node_spec.keys())[0]
         if len(node_spec) == 1:
             return (next_key, node_spec[next_key]), {}
-        # If the next value is a list, expand it using the default combinator if possible
-        if not self._is_combinator(next_key) and isinstance(node_spec[next_key], list) and self._default_combinator:
+        # If the next value is a list (but key is not a list), expand it using the default combinator if possible
+        if not self._is_combinator(next_key)\
+                and (isinstance(node_spec[next_key], list) and not self._is_literal(next_key))\
+                and self._default_combinator:
             return ('{}{}'.format(self._prefix(COMBINATOR), self._default_combinator), node_spec), {}
         next_node_spec = {k: v for k, v in node_spec.items() if k != next_key}
         return (next_key, node_spec[next_key]), next_node_spec
@@ -316,6 +329,19 @@ def _deghost(prop):
             raise ValueError('Cannot deghost a non-ghost property')
         return prop[1:]
 
+    @staticmethod
+    def _is_literal(prop):
+        return isinstance(prop, str) and prop.startswith(LITERAL)
+
+    @staticmethod
+    def _deliteral(prop):
+        if not SpecificationNodeParser._is_literal(prop):
+            raise ValueError('Cannot deliteral a non-literal property')
+        try:
+            return json.loads(prop[1:])
+        except json.JSONDecodeError:
+            return prop[1:]
+
     @staticmethod
     def _prefix(name):
         return '{}:'.format(name)

diff --git a/spawn/specification/specification.py b/spawn/specification/specification.py
@@ -529,7 +529,7 @@ def evaluate(self):
 class SpecificationNodeFactory:
     """Factory class for creating :class:`SpecificationNode` objects
     """
-    def create(self, parent, name, value, path, ghosts, children=None):
+    def create(self, parent, name, value, path, ghosts, children=None, literal=False):
         """Creates a :class:`SpecificationNode`, based on the value
 
         :param parent: The parent :class:`SpecificationNode`
@@ -542,15 +542,17 @@ def create(self, parent, name, value, path, ghosts, children=None):
         :type ghosts: dict
         :param children: The children of the new node, if any
         :type children: list
+        :param literal: if True, the value is not expandable and is set literally
+        :type literal: bool
         """
         children = children or []
         validate_type(ghosts, dict, 'ghosts')
         validate_type(children, list, 'children')
-        if isinstance(value, dict):
+        if isinstance(value, dict) and not literal:
             node = DictNode(parent, name, value, path, ghosts)
-        elif isinstance(value, list):
+        elif isinstance(value, list) and not literal:
             node = ListNode(parent, name, value, path, ghosts)
-        elif isinstance(value, ValueProxy):
+        elif isinstance(value, ValueProxy) and not literal:
             node = ValueProxyNode(parent, name, value, path, ghosts)
         else:
             name_index = self._index(name)

diff --git a/tests/parsers/specification_parser_tests.py b/tests/parsers/specification_parser_tests.py
@@ -15,9 +15,9 @@
 # along with this program; if not, write to the Free Software Foundation,
 # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301  USA
 import pytest
+import json
 
 from spawn.specification.generator_methods import *
-from spawn.specification.combinators import *
 from spawn.parsers.specification_parser import *
 from spawn.specification.specification import *
 from spawn.specification.value_proxy import *
@@ -724,3 +724,67 @@ def test_spec_format_error_raised_for_invalid_description(parser, description, e
     with pytest.raises(SpecFormatError) as e:
         parser.parse(description)
     assert str(e.value) == 'Invalid spec format: {}'.format(error)
+
+_literals = [
+    [4, 5, 6],
+    {'delta': 1, 'epsilon': 2},
+    '$NotAMacro',
+    '@NotAGenerator',
+    '#NotAnEvaluator',
+    '!this is not an equation'
+]
+@pytest.mark.parametrize('literal_value', _literals)
+def test_key_literal_properties_are_not_expanded(literal_value):
+    root_node = DefaultSpecificationNodeParser().parse({
+        'alpha': {
+            'beta': ['egg', 'tadpole'],
+            '~gamma': literal_value
+        }
+    })
+    root_node.evaluate()
+    expected = [
+        {'beta': 'egg', 'gamma': literal_value},
+        {'beta': 'tadpole', 'gamma': literal_value}
+    ]
+    properties = [l.collected_properties for l in root_node.leaves]
+    assert expected == properties
+
+@pytest.mark.parametrize('literal_value', _literals)
+def test_value_literal_properties_are_not_expanded(literal_value):
+    root_node = DefaultSpecificationNodeParser().parse({
+        'alpha': {
+            'beta': ['egg', 'tadpole'],
+            'gamma': '~' + (literal_value if isinstance(literal_value, str) else json.dumps(literal_value))
+        }
+    })
+    root_node.evaluate()
+    expected = [
+        {'beta': 'egg', 'gamma': literal_value},
+        {'beta': 'tadpole', 'gamma': literal_value}
+    ]
+    properties = [l.collected_properties for l in root_node.leaves]
+    assert expected == properties
+
+def test_with_literal_key_and_value_value_is_unchanged():
+    s = '~I just like tildes OK!'
+    root_node = DefaultSpecificationNodeParser().parse({
+        '~alpha': s
+    })
+    root_node.evaluate()
+    expected = [{'alpha': s}]
+    properties = [l.collected_properties for l in root_node.leaves]
+    assert expected == properties
+
+def test_multiple_literal_lists_in_object_does_not_combine():
+    root_node = DefaultSpecificationNodeParser().parse({
+        'alpha': {
+            '~beta': ['egg', 'tadpole'],
+            '~gamma': [1, 2, 3]
+        }
+    })
+    expected = [{
+        'beta': ['egg', 'tadpole'],
+        'gamma': [1, 2, 3]
+    }]
+    properties = [l.collected_properties for l in root_node.leaves]
+    assert expected == properties