Architectural Considerations for Language Versioning for the Web

 

Editors: Larry Masinter (Adobe), Jonathan Rees (Creative Commons)
Date: 26 August 2009
Latest version: http://www.w3.org/2001/tag/doc/versioning-html
This version: http://www.w3.org/2001/tag/doc/versioning-html/versioning-html-20090826
Previous version: http://www.w3.org/2001/tag/doc/versioning-html/versioning-html-20090611
Status: This is an editor's draft produced as part of the W3C Technical Architecture Group's work on ISSUE-41.

 

Status

(unfortunately, this document is still pretty rough, and there are many comments that have not been reflected in the content. The Terminology and Framework section is the most important to review now, and also References. This document looks like it will have a reading not only on versioning but also on distributed extensibility for HTML, though.)

Introduction

 

This document provides a framework for discussing versioning, evolution and extensions of languages and interfaces, with an aim toward providing guidance for W3C Working Groups considering whether and how to indicate versions, language changes, how to design extensible languages, provide for future extensibility, and deal with backward compatibility.

 

The W3C TAG has spent many years looking at versioning in general, and there are a number of drafts for which consensus has been elusive. One important motivation for continuing this work has been to specifically look at the issues and requirements for versioning in HTML, between HTML4 and HTML5.

 

W3C attempts defines languages. Languages evolve in many ways, but frequently evolution creates a new "versions" of a language. Evolution may be small and incremental or quite drastic. After evolution, however, new implementations may encounter older content, and new content may be offered to older implementations. The question is how to arrange for those encounters to "work", behave appropriately. How should language designers plan for evolution, and advise implementors of consumers and producers (creators) of content about how to "future proof" what they are making? And what is the role, if any, of a "version indicator" in such planning?

 

Background

 

The history of TAG work in versioning is summarized in Appendix I. More recently, the TAG reviewed the history of previous work: http://lists.w3.org/Archives/Public/www-tag/2009Apr/0028.html , and updated ISSUE-41 http://lists.w3.org/Archives/Public/www-tag/2009Apr/0061.html.

 

Terminology and Framework

 

This section lays out a framework for describing communication and the role of languages in them.   The document uses some common terms in ways that are very specifically defined for this framework, although the definitions attempt to be consistent with common usage. Terms defined specially here are in bold, while other terms of interest but assumed to be defined elsewhere are in italics.

 

The World Wide Web is a communication system; a way of using a network (and in particular the Internet) to communicate. Communication usually involves a protocol in which documents are sent from one agent (the producer) to another agent (the consumer).

 

Agent

 
 
 
An agent is software that is involved in communication; for example, a web browser, an authoring tool, a web server are all agents (A user-agent is an agent under direct control of a user.)
Producer
A producer is an agent that generates documents for transmission to one or more consumers. In fact, the production of a document and its communication may involve several steps, where the document is initially generated by one agent (for example, an editor agent controlled by a user), and then subsequently transmitted via a protocol to another agent.
Consumer
A consumer is an agent that performs some action when presented with a document sent via a protocol -- the consumer interprets the document. Note that a viewer, brower, or player are kinds of consumers, but so are search engines, translation gateways, summarizers, and other agents that receive and interpret documents.
 
Document
A document is (in this framework) a string of characters or bytes that might be transmitted from a producer to a consumer; a document is generally a instance of a language (or, in the case of compound documents, a combination of instances of the language). Note that this definition of document is, in some ways, intentionally broad, and includes any communication that might be sent in a HTTP response or an email message.
Language
In this framework, a language is an agreement of syntax and meaning between agents in a community, such that the agents in the community are interoperable when interpreting documents in the language (or, in the case of compound documents, part of the document). Note that the use of language here refers to "language as spoken"--what is actually implemented and understood -- as opposed to a specification.
Community
A community is a set of agents (producers and consumers) that wish to communicate interoperably. Note that there may be several overlapping communities.
 
Specification
A specification is written (human interpretable) documentation of an agreement of a community that describes and defines a version (or several versions) of a language.  Generally a specification describes syntax and meaning associated with constructs of the syntax. Often a specification will describe (mandate etc) both a conservative language -- a definition and set of tests for whether a particular document is conforming, and which a conforming producer might produce, as well as advice or normative behavior for conforming consumers to exhibit when presented with documents, even when they are not valid -- in this case, it is defining error behavior. The introduction of error behavior is not only important for consumers for graceful use, but is an important element of language extensibility (as described below.)
Version
A specification version is a particular specification belonging to a series or family of closely related specifications. A language version is a language (as implemented) which generally corresponds to a specification version. Note that specifications can be updated to correct mistakes, which might create a new specification edition but not a new version. Note that this definition of version actually encompasses any change from one language or specification, not ones that are associated with major versions.
Feature
A language is often extended by adding features: new capabilities that consumers exhibit and that producers can put into documents.
 
Interoperable
When a producer sends a document to a consumer, and the consumer's behavior or interpretation of documents matches -- to some degree of satisfaction -- the producer's expectation, the agents are said to be interoperable. (Note that there may be degrees of interoperability.) In addition, if two consumers interpretation of documents match satisfactorily, the two consumers are said to be interoperable. Two implementations of a specification are interoperable implementations if they are both interoperable (in either sense) but are also interoperable in a way that conforms to the specification.
 
 
 
Valid
A document is valid according to a specification if it matches the allowed syntax of the specification. (This is sometimes also called conforming)
Conform
Many specifications contain normative requirements for different classes or roles of agents, as well as requirements for valid documents. An agent is a conforming implementation if it meets those requirements.
 
 
Language- or Version-indicator
A language-indicator or version-indicator is either (a) a syntactic component of a language for inclusion within a document, or (b) a protocol element, transmitted outside of a document; the indicator identifies a specification (or family of specifications or version within that family). A producer supplies an indicator for use by the consumer of the document. Note that things like MIME types, DOCTYPE, XML namespaces and the like are all examples of indicators. This definition and the use of version indicators is expanded below.

 

Standards, Specifications, and Language Versions

 

 
 
 
The standards process is way of evolving specifications to allow common and interoperable changes to implementations of agents. The standards process involves bringing together a community (of document producers and of implementors of producer and consumer agents) to produce a specification that they all ("consensus") agree to implement and use. Insofar as a specification represents that agreement and it is followed, a good specification will improve interoperability, and thus increase the satisfaction of its users.

 

Reasons for Language Changes

 

It is difficult to come up with any language (computer or otherwise) that has not evolved, been extended, or otherwise "versioned" as long as the language has been in use. This history of extension and change applies to network protocols, character encoding standards, programming languages, and to every known technology found on the web. Among these, is difficult to come up with any case where a language hasn't gone through at least some minor incompatible change.

 

Why do languages evolve, in ways that might need to be called out as separate versions? Specifically, why might HTML evolve? How can we define how we define HTML5 today such that, if problems are discovered that require incompatible language changes, we can increase the interoperability of new implementations of receivers and producers and new documents and old implementations of receivers and producers and old documents.

 

It is not possible to guarantee that no incompatible changes will be needed in the future. There are several reasons why languages and their specifications need changes:

 

 

  1. Additional requirements: Users and implementors want the language to support some new feature that hadn't been thought of at the time of the design of the language or its previous versions. Some user agent requirements can be accommodated without actually changing the language, but often a language change is needed.
  2. Changed requirements: Original misunderstand of requirements or mismatch with practice as it is deployed.
  3. Difficulties are uncovered after specification is published: The standards process balances timeliness against safeguards for specification quality. For example, W3C CR requires two interoperable implementations. But, of course, two iimplementations aren't representative, and the requirements for those implementations are not exhaustive. Perhaps some agent contexts will require additional features or changes to existing features..
  4. Ambiguities appear: During development of a specification, implementors get together and write a specification that they believe they can implement. They're happy because the specification matches what they implement -- or so they think! However, all of the implementors were part of the specification development process and perhaps there are some things they know and agree on that aren't part of the specification. Later, some other implementor comes along and implements the specification as written, but, either because of confusing wording or missing information, their implementation is incompatible. Later there's a desire to update the specification to resolve the ambiguity, but acknowledge that this is a new "version".

 

 

Version indicators

 

As above, version indicator identifies to a consumer something about the nature of the document and/or how it is meant to be interpreted. For example, an indicator might imply that certain features are or are not used, or it might imply that certain interpretations of constructs are allowed or not allowed. An indicator might be associated with some named version of a language, which would usually mean that to find out what might occur and what it might mean, one should consult the specification for that version.

 

How new versions of languages might be indicated

 

There are many ways in which the version or intended interpretation of a document might be indicated. This section enumerates the kinds of version indicators available generally (out of band, in-band global, in-band local) and specifically for HTML  (MIME types, comments, DOCTYPE, new tags, namespaces).

 

Version indicators can either be

 

 

In-band version indicators can either be

 

 

Nature of Language Changes

 

Languages can change through

 

 

Because languages are contracts involving multiple parties, it is important to recognize that a language sets constraints on behavior of consumers. A language change can relax or constrict the freedoms of the consumer, e.g., make a previously conforming or interoperable consumer non-conforming or non-interoperable. Consumers can be permitted to do things with documents they weren't allowed to do before, e.g., can perform new behavior with a document that previously had to be treated as an error), or prohibited from doing something that were allowed before (e.g. considering something to be an error).

Use of version indicators as part of communication between producers and consumers (distribution)

Whether language changes can be recognized without version indicators depends on the type of change: Some augmentations might be recognized by appearance of syntax that wasn't previously recognized (i.e., the "version indictor" is the use of the feature itself). Augmentations might be ignored or merely processed incorrectly by old implementations rather than being recognized as intended with a formerly unimplemented interpretation. Restrictions, clarifications, incompatible changes cannot readily be determined by scanning a document, however. That is, a consumer that is aware of a new version, presented with a document without a version indicator, cannot readily determine which interpretation was intended by the document as the document syntax itself is consistent with both the previous and new version.

As has been seen with MIME types, external metadata is subject to risks of separation, lack of control by authors on deployment separation. Right now, new HTML features seem to be deployed on the web by HTTP servers "sniffing" the User Agent identifier (consumer capability) and using it to determine which edition of a HTML page should be generated for that consumer. This process is subject to some significant failures, mainly because new or otherwise unrecognized consumers have no way of indicating to such sniffers that they, too, intend to interpret the same features as another consumer.

Version indicators are also useful when a old consumer (user agent) encounters new content, in some cases. For this to occur requires extensibility features be implemented in the consumer in preparation for the possibility of new versions. This is usually accomplished by specifying (in a specification version) an error behavior which detects some syntactic construct, and also requiring that the consumer exhibit error behavior which informs the user (in the case of a user-agent consumer) or otherwise behaves in a way that uses the version indicator to identify that the document is an instance of a new or different version or language.

Utility of Version Indicators outside of distribution

 

One use case for embedded version indicators is to track versions during authoring, production and deployment of documents before they are communicated. Authors and authoring tools may well know which version of a language they are editing or producing content for, which features they are assuming and so forth. Without any way of marking the intended version in the content itself, it is likely that version indicators will be carried outside, and subject to loss. We need to consider the use cases of language version management during pre-publication processes, and also the use case of "browser version" sniffing and the failure cases.

Problems Version Indicators Cause

People put in the wrong one, translation tools use the latest version when they shouldn't etc.

Motivation of Implementors of agents (Formalism)

Situation: a producer generates a document containing parts that are not understood by the consumer. (The case that inspires the following discussion is the tension between experimentation with features such as RDFa and the "canonical" link type, and the "must understand" philosophy.)

 

In the formalism (below) we distinguish between points in producer/consumer payoff space with good (positive payoff), punishing (negative payoff), or neutral (zero payoff) coordinates along the producer/consumer axes.

 

The payoff to the producer will shape the producer's behavior, and the payoff to the consumer will shape the consumer's behavior.

 

The payoff to the producer depends on what the consumer does (unless communicating just "feels good" or is required by law), but will often be indirect. E.g., in advertising, it's not the payoff from any particular transaction that matters, but the amortized payoff from many transactions. When a text is broadcast to multiple audiences with different capacities, it matters a lot whether the producer knows that this is the case.

 

Example: Creators of good children's TV shows (which I hereby define to be the ones I like to watch) know that there are two audiences and craft their material so that both appreciate the content. Creators of bad ones don't and only aim for one audience. But including material meant only for grownups (positive consumer 2 payoff), perhaps disguised or deemphasized so as not to make those who don't understand anxious (negative consumer 1 payoff), is, to me at least, the essence of craft and quality.

 

We have the same situation with web content creators:

 

 

Those exercising craft have a more difficult job in creation and testing - they have to think - and this extra effort will only be made if the perceived benefit outweighs the cost. In a sense material that is knowingly destined for different audiences constitutes multiple communications channels, and the question of server/client compatibility (payoff) might be better thought of not as a language extensibility problem but a multiplexing problem.

 

If in-line language extensibility (think: child-inaccessible puns) is outlawed, the new material will be communicated *somehow*. (This is similar to the rewrite-based extensibility question in programming languages. Macros happen whether a language spec supports them or not; it's just a question of how extensibility will be managed - forking new languages (think: content-types), external preprocessors, or in-language macro facilities.)

 

It's not clear what purpose version markers in HTML, indicating the presence of stuff not understood by some clients, should have for clients that don't understand that stuff. I guess it could lead to "upgrade your browser" or "get this plugin" dialogs, or "save this file or choose application", the payoff of which is the ultimate benefit minus the annoyance of having to deal with the dialog. But in some cases knowing that you don't understand will only lead to consumer anxiety.

 

Analogy: Suppose I go to Peru and am spoken to in Spanish, which I don't understand. How can I tell how important what's being said should be to me? Sometimes very, sometimes not; sometimes I can tell whether it has must-understand status, sometimes I can't; in the latter case I only have anxiety (perhaps a lot, if it's an armed soldier who's speaking) and in a sense I might prefer to have heard nothing, like the child who happily doesn't know that an adult pun has just flown past in their TV show.

 

Adoption of these recommendations

 

Any decision to require or encourage use of version indicators as a way of modulating behavior will require some agreement of current browsers to do work that will only pay off in the future, and getting that agreement requires buy-in by the affected parties. However, we should not start with the presumption that “they won’t go along” without first making the case for why allowing for future non-compatible extensions in current browsers is good practice, even when such changes should be avoided if at all possible.

 

Specific HTML recommendations

 

There is still some question about whether, even if the case is made for a global in-band version indicator for HTML5, whether that should be a DOCTYPE or another different kind of indicator. There are some arguments made (needed to be examined) that current deployed browsers will not respond appropriately to new DOCTYPEs for HTML5, and so some other version indicator might be recommended.

 

Acknowledgements

Conclusions

 

References

 

 

 

Appendix I: Review of past W3C TAG work on versioning

 

  1. Noah Mendelsohn, "Summary of status of TAG issue xmlVersioning-41", March 2009 http://lists.w3.org/Archives/Public/www-tag/2009Mar/0146.html
  2. Jonathan Rees, "Alternative language versioning formalism", May 2008 http://lists.w3.org/Archives/Public/www-tag/2008May/0155.html
  3. Ian Jacobs and Norman Walsh, Architecture of the World Wide Web, Volume One, section 4.2.1. Versioning, December 2004 http://www.w3.org/TR/webarch/#versioning
  4. Noah Mendelsohn, "Version Identifiers Reconsidered", December 2007 http://www.w3.org/QA/2007/12/version_identifiers_reconsider.html
  5. John Kemp, "Formulate erratum text on versioning for the web architecture document", February 2009 http://lists.w3.org/Archives/Public/www-tag/2009Feb/0147.html
  6. David Orchard, "Extending and Versioning Languages: Compatibility Strategies", September 2008 http://www.w3.org/2001/tag/doc/versioning-compatibility-strategies
  7. Larry Masinter, "Report back on the TAG's work on versioning wrt HTML", February 2009 http://www.w3.org/html/wg/tracker/actions/108

 

Appendix II: Use of Version Indicators in Other Languages

 

It is empirically true that one can version a language without having inline version indicators. For example, Algol 60 and Algol 68 do not have version indicators. Supplying version indicators is a design choice. Algol is not really relevant, because people who operated Algol compilers had the power to tweak the programs. The Web is a completely different environment, because users of browsers can't practically tweak the input the browser gets from a site.

 

TBD: Survey

Review the compatibility and development workflow strategies for using different kinds of version indicators (future content with current readers,  distinguishing current from future content with future readers) http://lists.w3.org/Archives/Public/www-tag/2009Apr/0064.html

Appendix III: Versioning Formalism

 

JAR's attempt at a formalism for language compatibility. First published here:

 

http://w3.org/mid/063031F1-1645-4A4C-A350-2DF0077B9722@creativecommons.org

 

This framework was inspired by a paper on animal communication by Peter L. Hurd (J. Theor. Biol. 174:217-222 1995 [3]).

 

Two agents, a producer and a consumer, are playing a game that goes as follows:

 

  1. An objective o is chosen from O = a space of possible objectives
  2. Given o, the producer's (sender's) choice of text is via a function S: A->M where M = a space of possible texts (messages, strings)
  3. Given m, the consumer's (receiver's) choice of action is a function R: M->A where A = a space of possible actions (meanings, interpretations)
  4. Given o and a, success is judged according to a success (or "payoff") criterion Z(a,o). I.e. if Z(R(S(o)),o) then communication has been successful.

 

A simple situation would be where the objective space is simply the action space, and the text specifies the desired action:

 

Z(a,o) iff a = o.

 

Note: M = text space includes all possible texts, including those that are not used for communication.

 

Note: A = action space includes all possible actions, not just those that might be achieved through communication. Examples of actions: producing a certain visual display of the information in a hypertext document; generating the computational result specified by a computer program; doing nothing; rejecting the text.

 

The functions S and R are not uniquely determined by A, so the producer and consumer will need to agree on a correspondence. I'll define a "language" to be a contract that might be entered into between a producer and a consumer, presumably for the purpose of maximizing payoff to both producer and consumer.

 

The simplest kind of language would be to agree on the particular function R that is to be implemented by the consumer. Then given an objective o the producer can choose any text for which Z(R(S(o)),o).

 

However, we are interested in language extensibility. A language defined to merely specify the behavior of R cannot be extended because the consumer cannot change the interpretation of any text for fear that a producer unaware of the change might send it, in which case there would be no way to guarantee that R's action would meet the objective. Therefore we consider languages in which some texts are reserved for future expansion (i.e. sent texts are limited):

 

[[JAR 2009-06-11: I need to tweak the terminology and presentation, since the following definition of "language" is not consistent with the previous ones. I like the previous ones better so will need to rework the following.]]

 

definition: A language version L is a pair (F,I) where

 

 

The texts that are in M but not in F are extension points. Note that I defines actions for extension points. This will come in handy later.

 

definition: A producer produces only L if the image of its generation function (S, above) is a subset of F.

 

definition: A consumer implements L if R(m) = I(m) for all m in F.

 

(The producer will generally choose to further constrain S in order to maximize success.)

 

Now consider a language change - a transition from language version L to language version L'.

 

definition. L' is backward compatible with L iff any consumer that implements L' also implements L.

 

Put more simply (trivial theorem): L' is backward compatible with L iff I'(m) = I(m) for m in F - it's the same except at extension points.

 

We would like to also have some notion of forward compatibility: Any producer of L' would be satisfied with what an L-consumer does with its texts. Since a consumer that implements L cannot know how in advance what new texts (in F'-F) are supposed to mean, we have a problem: what should R(m) be when m is new?

 

To answer this, we introduce the notion of adequate defaulting. The idea is that there might be communication success if we substitute some weaker action â for the unknown future desired action a. In this situation we write â ~> a. To simplify the formalism we also say a ~> a for all a in A.

 

definition: A consumer respects L iff R(m) ~> I(m) for all m in M.

 

A consumer that implements L respects it, but not necessarily vice versa.

 

We can define forward compatible as follows: L' is forward compatible with L iff any consumer that respects L also respects L', i.e. R(m) ~> I(m) implies R(m) ~> I'(m) for all m in M. Trivial theorem: Forwards compatibility holds iff I(m) ~> I'(m) for m in M.

 

definition: L weakly extends to L' iff L' is backward and forward compatible with L, i.e. F' superset of F I(m) = I'(m) for all m in F I(m) ~> I'(m) for all m in {F' - F}

 

In order for forward compatibility to be transitive, we also need to make sure nothing happens with non-final texts to break future extensibility:

 

definition:. L extends to L' iff it weakly extends to L' and preserves or improves defaults defined by L:

 

 

Def. L is extensible if there is an L' that extends it.

 

Note. If ~> is transitive (is a partial order), then extends and the other relations are all transitive. [requires proof?]

 

 

 

Example:
  • A = O = {stop, go, caution, reverse, unassigned}
  • M = {red, yellow, green}
  • unassigned ~> o for all o in O
  • L = (F,I), L'=(F',I')
  • I = {green:go, stop:red, yellow:unassigned}
  • F = {red, green}
  • I' = {green:go, stop:red, yellow:caution}
  • F' = {red, green, yellow}

 

We want to be able to "kill off" a text - to decide in a future extension that it shouldn't have any meaning and shouldn't be sent. We can specify I(m) = k with Z(k,o) always false, and then the producer won't send it. This only works if either

 

  1. k is not on any "upgrade path" to an action that succeeds (i.e. not k ~> a), or
  2. k is not upgradable (k is in F).

 

(2) puts fewer constraints on A - it requires only one special undefined action, e.g. u as in the example, instead of two that behave differently in the partial order. However, (1) is better formally as one simply specifies a ~> k for any a, and then one may upgrade any text's action from a to k. (2) would require a change in some of our definitions to allow the meaning of a text to pass from a default action a to k, which is not related to it by ~>. (Maybe we could introduce a set K of killed texts as a component of a language.)

 

Note: If an action can only be done using a defaulted text, why would a producer not "cheat" by sending a text outside F in order to achieve that objective? We can remove the temptation by making sure that a defaulted objective is always achievable by a non-defaulted text: For all m, if Z(I(m),o), then there is some m' in F such that Z(I(m'),o).

 

Correspondence with [1]: final = in defined set OR undefined by virtue of having been "killed"; accept set = text's action succeeds for some objective

 

Enrichments:

 

We could extend the framework to nondeterministic producer and consumer behavior, to quantitative payoffs, and/or to differential producer/consumer payoffs. Evolutionary biologists do these things in order to explain the presence and absence of cheating in natural communication systems. There is probably no need to do this in a protocols and formal languages engineering setting, except by way of explaining why a producer would choose a richer action when a default would suffice.

 

[[JAR: I have revised my view on the importance of both differential payoff and nondeterminism. Given the difficult and contentious nature of the issues we're facing, I think it will pay off (so to speak) if the presentation were be changed to recognize these phenomena.]]

 

We should be able to model distributed extensibility in this setting: how precoordination (social agreement on how to split up the extension space) can enable the existence of upper bounds among the compatibility relations.

 

Thanks to Alan Bawden for checking my math. Any remaining errors are mine. - Jonathan

 

[1] http://www.w3.org/2001/tag/doc/versioning (terminology)
[2] http://www.w3.org/2001/tag/doc/versioning-strategies (strategies)
[3] http://scholar.google.com/scholar?hl=en&lr=&cluster=11821292421815354248 (Hurd's paper)

 

See also < Coordination Coordination_game wiki en.wikipedia.org http:>, http://www.stern.nyu.edu/networks/325.html

 

Appendix IV: Other considerations

 

This section is for things that haven't been integrated into the main doc.

 

This choice (?) has profound consequences for the design of future versions. Suppose that an A (old) text is marked with indicator A. #1 does not in itself imply that a text generated by an A-interpreter will lead to the desired payoff for a B producer, for any text. That will only be true if when we designed the versioning regime we made a stipulation that all future versions will have this property (new producers "must be" happy with what old consumers do with all texts). If we stipulate only sense #1, then future version designers do not have the freedom to transition a given interpretation of a given text from acceptable (in A) to less acceptable (in B) - or vice versa.

 

If there is in fact forwards and backwards compatibility in a language series, there may be no strong need for a version indicator, other than as a convenience (so that agents who care don't have to scan the document to see if it contains constructs it doesn't understand).

 

So in any discussion, you need to be clear about the sense of the version indicator.

 

Sense 1 is economical in that a consumer can always just use a B-interpreter to interpret according to language A. There is strong incentive for a consumer to assume it even when doing so isn't in spec. Sense 2 is harder to implement since the consumer needs two interpreters or two interpretation modes, one for A texts and one for B texts.

 

Version indicators can be helpful, but they just push off the problem one level - they are really part of the language(s) in question, so they have to be evaluated according to exactly the same criteria that one would apply to a language series that doesn't have them. Suppose you have language versions A and B, and then a "sum" language C = A + B whose texts consist of a version indicator followed by a text of either A or B. (If A and B both already have version indicators you *may* be able to take C texts = A texts union B texts.) You still have to agree ahead of time - before language B is invented - on how to interpret texts of C - that is, everyone concerned needs a priori knowledge of how to parse and understand version indicators, even if it's just to say that rejecting unknown versions, or unknown texts, is OK. When you design a language series initially, you may set aside a place for version indicators, and specify that the indicator "sublanguage" is extensible (i.e. new indicators may come along). If you get the indicator language wrong in the first place, e.g. if you define it to specify sense 1 instead of sense 2 or vice versa, then you may find yourself stuck, either underconstraining the series (so that old consumers can't consume new content with confidence) or overconstraining series (so that new content will be rejected by conforming old consumers).

 

So version indicators only support extensibility (or whatever other goal you're after) if the future consequences for both old and new consumers are articulated and documented before the whole process gets started.

 

Notes: Saying that C = A + B where B is not yet invented is not an nonsensical as it sounds. An extension may be thought of as a secret that is somehow known in principle, but not revealed to producers and consumers until some future date. I think of versioning and extension as being similar to the concept of single assignment or "future" in programming languages.

 

2. JAR used a different definition of "language" in his formal writing... I think that language (or language version) as class/predicate of interpreters, or equivalently requirements/specification/constraints on interpreters, is probably a more useful definition that either language as set of strings or language as single interpretation function on set of strings.

 

Languages can and have been evolved without changes that require implementations to implement forks to consume content from different versions of the language in different ways. On the Web, for example, URIs have evolved that way, and with some unnecessary exceptions, so have CSS, HTML, and the DOM APIs.

 

In fact, in the case of CSS and HTML, the only versioning has been quirks vs standards mode, a versioning that wasn't sanctioned by the specifications contemporary to its introduction, and which would have been unnecessary had the deviations from the original design required by deployed content been codified as standard, as we have been doing for the past few years with CSS 2.1 and HTML5.

 

Forking the language makes implementations orders of magnitude more complex. Watching the Internet Explorer engineers' pained expressions when one discusses the implications of their decision to ship multiple versions of their rendering engine makes this abundantly clear. It also makes the language less suitable for constrained devices (instead of one language to support, one effectively ends up with multiple languages to support), harder to test (instead of testing one language implementation, one effectively ends up testing multiple implementations, as well as their interactions in edge cases), and harder to document (instead of just specifying the weird behaviours that end up de-facto part of the language due to wide deployment of implementation bugs, one has to also specify the other behaviours expected in each version).

 

Language designers should strive to make their languages versionless at the syntax level.

 

Version indicators only support extensibility (or whatever other goal you're after) if the future consequences for both old and new consumers are articulated and documented before the whole process gets started.

 

That's not uniquely true of version indicators.

 

That's true no matter what technique is used to distinguish one version from another. The alternative, where there aren't any version identifiers, requires consumers to deal with both old and new markup as well.

 

For some languages and some applications, it may be reasonable to define a universal semantics for all versions, such as the HTML rule of ignoring wrappers it doesn't recognize. (Not that that hasn't introduced problems of its own, with special elements created over time just to work around the consequences of the "ignore wrappers" rule.)

 

For other languages and other applications, it may not be reasoanble to define a universal semantics. Applications must be expected in that case to do something else. Version identifiers offer a convenient mechanism to help users distinguish between versions, even if machines don't need them: "Unexpected element 'fribble' encountered in this V1.2.3 document. The element 'fribble' is not defined in V1.2.3."

 

To allow any particular input to be flagged as an error is itself (what I would call) semantics. We're having so much trouble with "error", "must accept", "must reject", "must understand", and so on, that it ought to be useful to deconstruct a bit and just talk about the desirability (payoffs) of these various outcomes for producer and consumer. I expect it to be helpful to treat outcomes such as reject, ignore, default, and understand uniformly, and talk about semantics (or specification) not as giving the single "correct" outcome for all consumers but as saying which possible outcomes are acceptable across consumers of varying abilities and inclinations. So ask not "should the consumer accept (or reject) X" but rather "should it be OK with the producer if the consumer rejects (or accepts) X".

 

Appendix V: Feedback on the June 1 draft

 

 

 

Version Detection: How consumers detect versions

(this is perhaps historical, need to fill out)

 

Even though it is possible to avoid having out-of-band or global-scope version indicators for augmentations, this does not mean that there are no advantages or uses for in-band global indicators. If there are multiple languages (whether Algol 60 vs Algol 68) or just multiple "modes", having a global-scope in-band version indicator allows for switching between one interpreter and another. Indicating the version in-band but requiring parsing of the text means that it isn't possible to evolve syntax or parsers.