Real-Time Test Oracles using Event Monitoring

Abstract: To gain confidence in that a dynamic real-time system behaves correctly, we test it. Automated verification & validation can be used to conduct testing of such systems in an effective and economic way. An event monitor can be used as a part of a test oracle to monitor the system that is being tested. The test oracle could use the data (i.e., the streams of events) derived from the tested system, to determine if an executed test case gave a positive or negative result. To do this, the test oracle compares the streams of events received from the event monitor with the event expressions derived from the formal specification, and decides if the executed test case has responded positive or negative. Any deviations between observed behaviour and accepted behaviour should be reported by the test oracle as a negative result. If the executed test case gave a negative result, the monitor part should signal this to the reporter part of the test oracle. This work aims to investigate how the event expressions can be derived from the formal specification, and in particular, how the event specification language Solicitor can be used to represent these event expressions. We also discuss the need for parameterized event types in Solicitor, and any other event specification languages used in event monitoring. We also show that support for parameterized event types is a significant requirement for such languages.