/ elib / distrib / captp

CapTP Ops: WormholeOp

WormholeOp(packets :byte[],
           source  :VatID,
           dest    :VatID)

If dest is the receiving vat, then it should try sending this packets data to itself as encrypted VatTP communications originating with source, processing sequence info so that redundant packets data are simply ignored. The receiving vat should process this data ahead of processing further requests from the sending vat.

If dest is not the receiving vat,, and if it currently has a live connection to dest or if it forms one while still connected to the requesting vat, then it should wormhole these bytes towards dest before allowing any further causality to flow from the requesting vat through the receiving vat to dest. Otherwise is can discard this data.

The Conflict Solved by WormholeOp

Without the WormholeOp, there are a set of requirements in the E semantics that are individually quite sensible and compelling, but that jointly seem to be impossible:

• Partial ordering: In a 3-vat Granovetter introduction, that the forked reference sent to Bob gives Bob access only to post-X Carol, only enabling messages from Bob to arrive at Carol after X is delivered. (Even without the WormholeOp, the current implementations of E meet this requirement.)

• Allow services with Near arguments. In order to allow services like the MintMaker to be correct when written this simply, we allow it to require its arguments, such as the src argument of the deposit message, to be Near. Of course, this requirement can only be satisfied when the argument Purse is in the same vat as the receiving Purse, but that is the case here for any valid argument Purse, so no problem. Further, remote clients need to be able to ensure that the argument as delivered is Near, even though the argument as sent cannot be. For this we adopt the argument passing rule "Going Home" that says a Far reference (a Resolved remote reference) sent as an argument in a message sent to the designated object's hosting vat arrives as a Near reference.

  In the absence of this requirement, since clients could not ensure that arguments arrive Near, the remotely invokable interfaces of the MintMaker would always have to deal with the possibility of Eventual arguments. It could wait on these arguments, and put the previous method body in a when/catch clause, but it's worse than that. In a deposit followed by a withdraw, the withdraw should not fail for insufficient funds if the deposit would have provided these funds, just because the deposit hasn't been scheduled yet. This would necessitate rather complex postponement logic in the MintMaker. But much of the goal of E is to enable simple secure distributed services to be coded simply. Therefore, this requirement seems necessary. (Even without the WormholeOp, the current implementations of E meet this requirement.)

• Preserve passability. A PassByCopy object that contains only passable parts should be passable by copy between vats. The problem here is a hashtable-based PassByCopy collection, T, (whether the current EMap or the anticipated Hydro-based replacement) that has as one of it's keys a PassByProxy object, let's say Carol in VatC. If T is passed to Alice in VatA, everything's cool, and it arrives with a Far reference to Carol as its key. However, if Alice were to further pass this to Bob in VatB, then, in order for T to still be operational at the moment of arrival, this key would have to arrive as a Settled remote reference to Carol, which is to say, a Far reference. (For reasons explained below, E won't do this until the WormholeOp is implemented. Instead, in current E implementations, the reference to Carol will arrive as an Unresolved remote reference (a RemotePromise), and therefore T will fail to unserialize. This is the Lost Resolution bug.)

The conflict arises when, in the Preserve Passability scenario, Alice had sent messages (like X) to Carol that hadn't yet arrived in Carol's vat when she sends T to Bob. The reference to Carol that Bob gets in T must be "behind" X, which would seem to make it different than other Resolved references Bob might have to Carol. However, since this new references is Resolved as well, if Bob includes it as an argument in a message to Carol's vat, it must arrive as a Near reference to Carol. However, because Near references give immediate access, it may not arrive as a Near references until all prior messages, such as X from Alice, have drained out.

Some Plausible Engineering Solutions

1. Invent the Indirect reference. Introduce a new kind of reference into our reference taxonomy: The Indirect reference is a Settled but not Resolved remote reference to a PassByProxy object. Dean, MarcS, and I (MarkM) actually worked out the semantics for this, and earlier in the CVS history you'll find some corresponding taxonomy diagrams and explanations, but it just made E too hard to explain.

2. Wait for drainage. Block all further communications from VatB to VatC until all communications from VatA to VatC prior to the introduction have drained out, or until the VatA/VatC timeout period expires, killing the connection. This could repeatedly impose a huge cost on VatB for the sake of only a few objects in VatB. While E avoids making any claims about resistance to denial of service attack, this would be too egregious a vulnerability to such an attack.

3. WormholeOp. When VatA introduces VatB to Carol, she first wormholes the unacknowledged portion of her outgoing VatA-to-VatC VatTP stream through VatB for delivery to VatC. VatB then wormholes it towards VatC before acting on further messages from VatA. VatTP already encrypts this traffic for secrecy, integrity, authenticity, and protection against replay attacks, so that this communications can be carried by untrusted intermediaries. VatB is just another untrusted intermediary. If VatC gets the next communication in the VatA/VatC stream via a wormhole from VatB, that's fine. It doesn't matter where the bits came from, as long as they pass all the crypto tests. When these same bits come in redundantly through another path, they may be safely ignored.

4. Inter-vat forks make new identities. Do (or adapt) what Droplets does, as described in the thread rooted here. Briefly, give each inter-vat fork of a reference its own new Resolved sameness identity, whether the original reference was Resolved or not at the time of the fork.

The WormholeOp needs to happen iff we decide to pursue path #3, which is the current expectation.

Wormhole Introduction Scenario

The Wormhole introduction is only needed when VatA sends to VatB a Far reference for an object on VatC. It isn't needed for any 2-vat introductions, and it isn't needed when the reference passed from VatA to VatB is Unresolved, such as a RemotePromise whose Resolver is on VatC, or is travelling to VatC.

*** To be written, but involves these CapTP steps:

VatA to VatC:

def vine := NonceLocator <- provideFor(farCarol,
                                       vatBID,
                                       nonce,
                                       carolSwissHash)

VatA to VatB:

WormholeOp(/*unacknowledged encrypted A-to-C traffic*/,
           vatAID,
           vatCID)
/*some message containing: */
... Far3Desc(VatCSearchPath,
             VatCID,
             nonce,
             carolSwissHash,
             vine) ...

VatB to VatC:

WormholeOp(/*unacknowledged encrypted A-to-C traffic*/,
           vatAID,
           vatCID)
def carolPromise := NonceLocator <- acceptFrom(vatAID,
                                               nonce,
                                               carolSwissHash,
                                               vine)

/* carolPromise, a RemotePromise, is then magically turned into farCarol, a Far reference. */

If VatB fails to deliver the wormhole data to VatC, then the acceptFrom message will either arrive too early and find no reference to Carol (fail safe), or arrive after the provideFor, and thus after the preceding messages to VatC (correct partial ordering).

So, if VatA and VatB are cooperative, they are both assured that the needed provideFor "from" VatA will be processed by VatC before VatC sees the corresponding acceptFrom from VatB. If either is uncooperative, they cannot cause damage beyond that accounted for by the object-level semantics. Because the data takes redundant paths, neither side will get stuck waiting on the other to timeout.