/ download / 0-8-25

Highlights of 0.8.25n

No Longer Eating Our Own Deprecated Dog Food Sugar

no-dot-call
explicit-result-guard

New Style Guard Expressions

Notice of Non-Upwards-Compatible Changes
Guard Parameters Use Square Brackets
New Guard Templates
Guarded Expressions
Proposal: We should require Guards and Auditors to be DeepFrozen
Proposal: Parameterized Interface Declaration Syntax
Open Questions

Further Guard Improvements

Improvements in Causality Tracing

Name Changes

Bugs Closed

No Longer Eating Our Own Deprecated Dog Food Sugar

We have ended our own use of some of our deprecated syntactic features:

• "e.enable.no-dot-call" is now set to "warn". We are no longer using juxtaposition for message sending: Rather than saying "bob foo(carol)", Alice must now say "bob.foo(carol)" or a warning will be output when her code is parsed.

? 2 add(3)
# warning: The optional e.enable.no-dot-call feature \
#          (see eprops.txt) is set to "warn".

# value: 5

? 2.add(3)
# value: 5

? pragma.disable("no-dot-call")

? 2 add(3)
# syntax error: The optional e.enable.no-dot-call feature \
#               (see eprops.txt) is currently off.
#   2 add(3)
#          ^

• In anticipation of switching to the easy-return style, "e.enable.explicit-result-guard" is now set to "warn". Currently, an absent result guard is equivalent to ":void". Once we switch to the easy-return style, a absent result guard will be equivalent to ":any". In order to survive the transition, from 0-8-25k up through 0-9, a result guard must be provided or at least a warning will be generated. Starting in the 0-8-25k release, our own code always uses an explicit result guard.

  ? def foo() {3}
  # warning: The optional e.enable.explicit-result-guard feature \
  #          (see eprops.txt) is set to "warn".
   
  # value: <foo>
  
  ? "" + foo()
  # value: "null"
  
  ? pragma.disable("explicit-result-guard")
  
  ? def foo() {3}
  # value: <foo>
  
  ? "" + foo()
  # value: "null"
  
  ? def foo() {^3}
  # value: <foo>
  
  ? "" + foo()
  # value: "null"
  
  ? pragma.enable("easy-return")
  
  ? def foo() {3}
  # value: <foo>
  
  ? "" + foo()
  # value: "null"
  
  ? def foo() {^3}
  # value: <foo>
  
  ? "" + foo()
  # value: "3"

As with "no-dot-call" and "explicit-result-guard", many of the properties controlling what the parser accepts can now be set to "warn". This can occur in properties files or on the command line, as you'd expect. In addition, for those properties (let's say "foo") that you could control by 'pragma.enable("foo")' or 'pragma.disable("foo")', you can now say 'pragma.warn("foo")'. The parser can now be parameterized with a TextWriter representing the text out stream on which to report these warnings.

Between the source tree and the erights.org website, we fixed over 4000 occurrences of these two deprecated features. To succeed at this, we mostly automated the process. The tool src/esrc/scripts/srccheck.e detects various problems in our source tree and reports the source position where they occur -- even including updoc scripts embedded as examples in html pages! The draft proto-tool src/esrc/scripts/srccheck-post.e was used to perform the automatic fixups of the problems reported by the first tool. I think it will grow into something useful, but right now is ideosyncratic and not fully automated. But it's interesting to read.

New Style Guard Expressions

Notice of Non-Upwards-Compatible Changes

• The "sturdy" guard has been renamed "Sturdy", in keeping with a simpler interpretation of our guard naming conventions.

• Up through the 0.8.25b release of E, you could express a Tuple guard by just writing a list of guards, for example:

  def fooUncaller(x) :[any, String, any[]] { ...

  This no longer works. (It was internally too kludgy to leave it in and simply deprecate it.) Instead, you need to use the Tuple guard template:

  def fooUncaller(x) :Tuple[any, String, any[]] { ...

  which brings us to the new style of guard expressions supported in this release:

Guard Parameters Use Square Brackets

An object, like Tuple above or nullOk used in the example below, that makes and returns a new guard when invoked with guard arguments is called a guard template. These correspond to the traditional notion of parameterized types and type constructors. In order to give guard expressions a different look than normal expressions, the argument list to a guard template is now written with square brackets instead of parentheses.

Whereas you used to say, for example

def barUncaller(x) :nullOk([any, String, any[]]) { ...

now you'd say

def barUncaller(x) :nullOk[Tuple[any, String, any[]]] { ...

The guard templates already present in the 0.8.25b release -- nullOk, vow, rcvr -- still accept their arguments using parens as well, but this use is deprecated and will be dropped. You should switch to square brackets.

Guard templates should usually also act directly as guards corresponding to their most permissive parameterization. A guard template should not interpret an empty argument list e.g., T[], as an empty template argument list, but rather treat this as equivalent to List[T], as explained below.

New Guard Templates

• List[T] - The expression List[T] forms a guard which coerces to a ConstList in which each element has been coerced by T. Guards should follow the convention that T[] is synonymous with List[T]. When used by itself as a guard, List is equivalent to List[any] or any[].

• Tuple[T1,T2,T3,...] - The expression Tuple[T1,T2,T3] forms a guard which coerces to a three element ConstList, in which the elements have been coerced by, respectively, T1, T2, and T3. Tuple cannot be used by itself as a guard.

• Map[Tk,Tv] - The expression Map[Tk,Tv] forms a guard which coerces to a ConstMap in which all keys have been coerced by Tk, and all values have been coerced by Tv.

• any[T1,T2,...] - The expression any[T1,T2] forms a guard which first tries coercing with T1, or, if that fails, then tries with T2. Guards should follow the convention that T1 | T2 is synonymous with any[T1,T2].

• all[T1,T2,...] - Bug: Not yet written, implemented, or even reserved.

Guarded Expressions

If you turn on the e.enable.cast property, then you can use guard syntax to the right of an expression, just as you can already do with a variable name definition or a method declaration.

? pragma.enable("cast")
? interp.setExpand(true)

? (3+4) :float64
# expansion: ValueGuard.coerce(float64, null).coerce(3.add(4), null)

# value: 7.0

As we see here, this has the effect of a Java-like cast expression.

Proposal: We should require Guards and Auditors to be DeepFrozen

In the expansion above, the further coercing of the presumed guard to a ValueGuard currently has no significant effect. The same coercion happens internal to the Kernel-E implementation for guards on variable definitions or results of methods -- they are coerced to ValueGuards first. This is in anticipation of a new requirement to be imposed once we have Auditors working: We should require Guards and Auditors to be DeepFrozen. I will raise this issue separately on the e-lang list.

Proposal: Parameterized Interface Declaration Syntax

Given the following minimally-typed fold function (familiar from the functional programming literature):

def fold(first, binOp) :any {
    ^def foldFunc(list) :any {
        var result := first
        for v in list {
            result := binOp(result, v)
        }
        ^result
}   }

how would we write a corresponding interface declaration for describing its type? The answers in E and Java as of this writing (0.8.25c) are unsatisfying. This is exactly the kind of example that type parameters were invented to describe.

The following syntax is now reserved in the grammar but not yet accepted:

interface BinOp[A,B] (a :A, b :B) :A

interface FoldFunc[Elem,Result] (list :Elem[]) :Result

interface Fold[Elem,Result] {
    to run (first :Result, binOp :BinOp[Result,Elem]) :FoldFunc[Elem,Result]
}

Using these declarations, and using yet more reserved syntax, we could perhaps write a fully typed version of fold as:

def fold[Elem,Result] implements Fold[Elem,Result] {
    to run(first :Result, binOp :BinOp[Result,Elem]) :FoldFunc[Elem,Result] {
        ^def foldFunc implements FoldFunc[Elem,Result] {
            to run(list :Elem[]) :Result {
                var result :Result := first
                for v :Elem in list {
                    result := binOp(result, v)
                }
                ^result
}   }   }   }

Which definition of fold is clearer? The tradeoffs on the spectrum between these two examples are obvious and inescapable. Once both are accepted, E will enable the programmer to choose where on this spectrum they wish their code to lie.

It remains to be seen how well automatic tools will be able to infer the equivalent of the second example from the first. (ML performs the equivalent of this inference, giving us hope. But ML is designed to enable this inference to succeed, whereas E is not.)

Parameterized types are a big deal -- the theory behind them (for example, Java1.5's use of F-bounded quantification) is way over my head. So I don't intend to have any notion of parameterized guards implemented in an E 1.0 time frame. However, I'd like to reserve the syntax needed to get to somewhere decent that also aligns with E's goals.

Open Questions:

• How does shifting from static type checking to dynamic soft-type checking (guards) affect the parameterized checking issues?

• E has much greater need to use soft types to express limits on authority, rather than the conventional guarantees on provided signatures. The first is an issue of partial correctness, or, in a security context, breach. The second is only an issue of guaranteeing progress, or, in a security context, avoiding a denial of service (even if it can't really do that). It seems conventional type checking is attempting to provide the less important set of guarantees -- lower bounds on what a component can do -- whereas, from both a correctness and a security perpective, it would be more valuable to guarantee upper bounds.

• If we shift our attention to using "types" to guarantee upper bounds, might the implied notion of subtyping instead be co-variant on both arguments and results?

• How do these notions of bounds relate to Fred Spiessens' investigations? Might we use the sub-language of guards and auditors to enforce the kinds of constraints he is working to formalize? I think this is where we will find the big payoffs for language-based support for security.

Further Guard Improvements

Now mostly implemented: __conformTo(valueGuard)

In the following example, foo's __conformTo method should really dispatch on the valueGuard parameter, and give this answer if it thinks this valueGuard would be satisfied by an integer.

? def foo {
>     to __conformTo(valueGuard) :any { ^3 }
> }
# value: <foo>

? 2 + foo
# value: 5

Terms for literal data types can now __conformTo

? def t := term`foo(3,4,5)`
# value: term`foo(3, 4, 5)`

? def term`foo(@jj*)` := t
# value: term`foo(3, 4, 5)`

? jj
# value: [term`3`, term`4`, term`5`]

? def term`foo(@{ii :int[]}*)` := t
# value: term`foo(3, 4, 5)`

? ii
# value: [3, 4, 5]

Without the guard on the jj variable, by the rules of ellipses, jj is bound to a list of int-Terms. But what if we just want a list of ints? Now that we can say what we want, we can get it.

Improvements in Causality Tracing

To be written...

Name Changes

• Data - Now synonymous with DeepPassByCopy
• stdin - Reserved; to be a sibling of stdout and stderr. But waits until we have EIO.
• jar__uriGetter - Added. Wraps the built in one. Needed to handle source position URL for sources obtained from jar files.
• twine__quasiParser - Added. Like simple__quasiParser, but preserves Twine annotations (alleged source position of origin). Used by the updoc parsers in order to report using source position info.

Bugs Closed

		Thanks to...
simple__quasiParser fails on empty quasi string (``)		Terry Stanley
Uninformative quasi-match failure messages
Trailing commas in lists		Jon Leonard and Kevin Reid
Ref.broken(null) causes misbehavior		Kevin Reid
Bogus "No current Vat" error (By fixing this, CapDesk's rename now also works.)		Terry Stanley and Marc Stiegler
javax.swing.event.ListDataEvent unusable		Kevin Reid and Marc Stiegler
Term tree grammar now accepts trailing commas in lists.		Kevin Reid
integer %% not behaving as documented.		Kevin Reid