Conducting
Verification & Validation following DO178B guidelines
Need for Standard
Because of the life criticality involved in the avionics software, it
became necessary to establish a uniform, consistent definition of criteria
for substantiating evidence for the absence of critical design errors,
answering the following questions:
1.How was it known that the testing was comprehensive and complete?
2.How was it known that the system requirements were comprehensive and
complete?
3.How was it known that the software requirements were comprehensive
and complete and interpreted the system requirement accurately?
4.How do we provide proof that a design or implementation error, which
may be present, cannot produce a safety critical situation?
Verification and Validation became new terms. Verification provided
the proof that a system was built to the requirements and validation
established that the requirements were complete and correct.
|
Key
information on DO178B
The verification of such a software as per DO-178B standards depends
on the level of criticality of the software. Software level is based
upon the contribution of software to potential failure conditions as
determined by the system safety assessment process. The software level
implies that the level of effort required to show compliance with FAA
(Federal Aviation Administartion) certification requirements varies
with the failure condition category.
The software level definitions are:
a. Level A: Software whose anomalous behavior, as shown by the
system safety assessment process, would cause or contribute to a failure
of system function resulting in a catastrophic failure condition for
the aircraft. For example software whose failure conditions would prevent
continued safe flight and landing.
b. Level B: Software whose anomalous behavior, as shown by the
system safety assessment process, would cause or contribute to a failure
of system function resulting in a hazardous/severe-major failure condition
for the aircraft. For example software whose failure conditions would
reduce the capability of the aircraft or the ability of the crew to
cope with adverse operating conditions to the extent that there would
be physical distress or higher workload such that the flight crew could
not be relied on to perform their tasks accurately or completely.
c. Level C: Software whose anomalous behavior, as shown by the
system safety assessment process, would cause or contribute to a failure
of system function resulting in a major failure condition for the aircraft.
For example software whose failure conditions would reduce the capability
of the aircraft or the ability of the crew to cope with adverse operating
conditions to the extent that there would be a significant increase
in crew workload.
d. Level D: Software whose anomalous behavior, as shown by the
system safety assessment process, would cause or contribute to a failure
of system function resulting in a minor failure condition for the aircraft.
For example software whose failure conditions would cause slight increase
in crew workload, such as, routine flight plan changes, or some inconvenience
to occupants.
e. Level E: Software whose anomalous behavior, as shown by the
system safety assessment process, would cause or contribute to a failure
of system function with no effect on aircraft operational capability
or pilot workload.
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
Objective |
|
Applicabilty by SW level |
Output |
|
|||||||||||
|
|
Description |
Ref.(DO-178B) |
A |
B |
C |
D |
Description |
|
||||||||
|
1 |
Test
procedures are correct. |
6.3.6b |
l |
m |
m |
|
Software
Verification Cases and Procedures |
|
||||||||
|
2 |
Test
results are correct and discrepancies explained. |
6.3.6c |
l |
m |
m |
|
Software
Verification Results |
|
||||||||
|
3 |
Test
coverage of high-level requirements is achieved. |
6.4.4.1 |
l |
m |
m |
m |
Software
Verification Results |
|
||||||||
|
4 |
Test
coverage of low-level requirements is achieved. |
6.4.4.1 |
l |
m |
m |
|
Software
Verification Results |
|
||||||||
|
5 |
Test
coverage of software structure (modified condition/decision) is
achieved. |
6.4.4.2 |
l |
|
|
|
Software
Verification Results |
|
||||||||
|
6 |
Test
coverage of software structure (decision coverage) is achieved. |
6.4.4.2a 6.4.4.2b |
l |
l |
|
|
Software
Verification Results |
|
||||||||
|
7 |
Test
coverage of software structure (statement coverage) is achieved. |
6.4.4.2a 6.4.4.2b |
l |
l |
m |
|
Software
Verification Results |
|
||||||||
|
8 |
Test
coverage of software structure (data coupling and control coupling)
is achieved. |
6.4.4.2c |
l |
l |
m |
|
Software
Verification Results |
|
||||||||
The
following verification documents supporting compliance to DO-178B need
to be produced:
1.Software Verification Plan
2.Software Verification Cases and Procedures ( Unit and Integration
test plan and procedures)
3.Qualification test procedures (Formal Evaluation Plan and Procedures)
4.Problem Reports
5.Software Verification Results
The planning of verification activities (like writing software verification
plan, various test plans) is mandatory for a project which follows DO-178B
standard. Also as illustrated in the verification effort table, based
on the level of the software, structural coverage is very stringent
when compared to a ‘regular’ project. Submission of problem reports
to the certification authority is a must.
In general, managing Verification activities poses lots of problems.
|
The planning and co-ordination of verification activities amongst the
team can be made easy and handled efficiently by using tools like SmartWorks. I used this tool inorder to achieve my objectives.
SmartWorks –Project Planner can be used to plan the entire verification process activities. The various activities like writing test plans (Unit test plan, integration test plan, Formal evaluation plan), performing code review, conducting the various tests as per their plans, generating reports…..can be created as tasks in Project Planner with task owner and the planned start and end date being assigned. Project Planner automatically sends a mail notifying the owners of their assignments. The document generated as a result of any task can be attached against the task. This allows the maintenance of the documents electronically. For example the test cases and procedure documents can be attached against its task and hence viewed by all the team members.
The
status of the task (new task, task completed) can be maintained in Project Planner.
The task owner is reminded with a mail if the task is not completed
when the planned end date approaches. This helps one to monitor the
project progress efficiently. Also whenever a change in plan occurs
(may be due to slippage, attrition or cost escalation) Project planner
recognizes that a risk had happened and suggests contingency plan from
it's database. This helps the team to know how many risks happened during
the course of a project and how effective were the contingency plans
in handling the risks. It allows you to import risk database from similar
projects done in the past by a company. This allows the project manager
to be well prepared for the risks which are likley to occur and quickly
take effective corrective action
The evaluator submits defects while conducting testing. This forms the
problem reports
At any point of time PM/PL can therefore monitor the project progress
and replan a task if need be. Project Planner allows one to replan the
task and associate the risk factor with it due to replanning.
Vasanthi heads the verification & validation team at Accord Software & System which is currently engaged in executing a project which adheres to DO178B standards. The project is concerned with evaluating a navigation equipment for flight worthiness and eventual FAA certification. In case you have any questions you may reach her at whitepapers@accord-soft.com