Earlier this week the WS-I voiced both its consent of the last versions of Basic Profile (BP) 1.2, BP 2.0, and Reliable Secure Profile (RSP) 1.0 and its own dissolution. There has been a bit of deliberation about the aptitude (or insufficient thereof) of these profiles and "what this all means". Rather than increase to that discussion, we wish to prominence an aspect of these profiles that we apprehension will be ignored - their testing.
Just to refresh; the WS-I profiles residence interoperability problems by nailing down, in the many solid conditions possible, the make up and essence of on-the-wire messages. The profiles do this in statements called mandate . The subsequent to e.g. is taken from BP 2.0.
R9981 An ENVELOPE MUST have precisely 0 or a youngster elements of the soap12:Body element.
Simple enough, but how do you assessment if an doing conforms to this requirement? You could, as formerly versions of BP did, write multi-part versions of a assessment focus using not similar SOAP stacks, assessment them against any other, and see if you wash out out any problems. Unfortunately, scold focus actions and scold on-the-wire actions are not 100% correlated. For example, suspect you are contrast two SOAP implementations, IA and IB, against a another. Suppose that, beneath a few circumstances, IA sends messages with two young kids in the SOAP Body (in breach of R9981). Further suspect that IB successfully processes these messages and the focus functions as expected. "No harm, no foul", right? Wrong. Remember the indicate was to assessment either IA and IB conformed to the profile. IA does not, but we have no way of knowing that since IB wasn't created to assessment form conformance.
Detecting necessity violations around application-level contrast doesn't work, in the broad case, since the WS-I profiles do not say anything about what an doing should/must do in the face of non-conformant behavior. You can never be certain that the implementations entangled in a assessment are not, as IB was doing in the example, accepting non-conformant actions (there are moreover other, more complex cases in that both IA and IB flop to heed to the profile). While it's superb if a set of implementations succeed to interoperate notwithstanding the fact that a or more of them are not adapting to the profile, its critical to comprehend that this interoperability is skilled around an implied "hidden agreement" that undermines the worth of that profile.
The BP and RSP working groups addressed this situation by taking advantage of a finer-grained contrast methodology in that the on-the-wire messages are prisoner and automatically analyzed using a set of assessment assertions (TAs). A assessment avowal is, essentially, a set of XPath 2.0 statements which, when run over a coming after of prisoner messages (a summary record ), will inform you if any of the messages in that record infringe the assertion. The subsequent to e.g. is the TA for R9981 (above):
;
;
/wsil:testLog/wsil:messageLog/wsil:message/wsil:messageContents/soap12:Envelope
;
; count(soap12:Body/*) le 1 ;
;
The pouch does not have precisely 0 or a youngster elements of the
soap12:Body
;
...
;
Looking at this example, you can obtain a few clarity of how TAs work. Messages are choosen from the summary record using the countenance in the ; component (in this case, every SOAP 1.2 Envelope in the summary log), and the countenance in the ; component is evaluated for true/false against these targets. Of course, I've chosen the simplest probable requirement/TA combo for my example. The assessment horizon used in BP 1.2/2.0 and RSP 1.0 can do things similar to relate solicit and reply messages, anxiety WSDL 1.1 definitions, bottom the analysis of a TA on the results of a formerly TA, etc.
Before we go any serve we must be give credit to Jacques Durand of Fujitsu for conceptualizing and building both the assessment analyzer horizon together with the majority of the TAs in BP 1.2/2.0 and RSP 1.0. It's moreover critical to indicate out that growth of this horizon is stability in the TAMELizer plan and the OASIS Test Assertions Guidelines TC .
To head off inconsistencies, the TAs for a form are placed in the XML source of the form itself. The routine of extracting these TAs and using them to investigate a summary record is described in the user guide that accompanies the analyzer in the particular assessment collection packages for BP 1.2/2.0 and RSP 1.0 . Rather than puncture in to the technical sum of that process, though, we longed for to share a few observations on the belongings of this draw close on the experience of both essay and contrast profiles. These belongings drop in to two buckets - those caused by the routine of essay and debugging TAs, and those brought about by the industrialisation that this sort of contrast allows.
The initial bucket of belongings dealt primarily with the showing and fortitude of vague requirements. An vague necessity is basically a no-op that doesn't upgrade interoperability, so it was utilitarian to mark such mandate and put together them as early as possible. Sometimes, when essay a TA for a requirement, it would turn coherent that a necessity was ambiguous. In other cases, the TA writer might appreciate a necessity in a particular way and subsequent contrast would trigger failures in this TA. The consequent scrutiny would detect the play on words in the underlying requirement. Finally, once we all had "TAs on the brain", the members of the BP and RSP working groups tended to write any new mandate in a more precise fashion that translated simply in to XPath.
The second bucket of belongings dealt primarily with the grade of assessment coverage. Anyone who has tested protocols is wakeful of the "I'm seeking for improper actions X, so I'll do something to bleed that behavior, then assessment for X" trap. Obviously the fact that you are perplexing to make X come about doesn't head off improper actions Y from happening, but there are often scalability problems that head off you from checking for all probable improper behaviors in every test. Assertion formed contrast helps to avoid this entice because, during the analysis of a summary log, every summary is a prospective aim for every TA. During the contrast of RSP a assessment designed to illegal errors in a area (e.g. WS-SecureConversation) would infrequently trigger TA failures in other area (e.g. WS-Addressing).
Finally, and many importantly, the use of avowal formed contrast lifted the club on the grade of conformance of the implementations that participated in the contrast compulsory to finalize the profiles. There was a time during the contrast of RSP in that any of the implementations transfered all of the application-level tests but, upon analyzing the messages exchanged during these tests, we found TA failures. Under the formerly "rules" for WS-I profiles we would have spoken feat and vanished home. Instead we either (a) prearranged the bug that caused the error, (b) prearranged the TA that wrongly reported the error, or (c) flagged the bug for a subsequent let go of the offending implementation. Without waxing too hyperbolic, we feel assured in reporting that the implementations that participated in the the contrast compulsory to finalize BP 1.2/2.0 and RSP 1.0 heed to those profiles to a larger grade than any WS-I profiles to date.
So we do not appear too gung-ho, we will indicate out that here are a number of problems with this kind of avowal formed testing:
No comments:
Post a Comment