System Tests

EasyNMEA provides a set of test which execute end-to-end verification of the library’s functionalities. This is done by simulating a NMEA device sending data to a serial port. This data is then received by a EasyNMEA application which uses the library’s public API to open the serial connection, wait until data of any given kind is received, and log this data for validation against expectations. For connecting the NMEA device double and the EasyNMEA application, socat is used to create a pair of virtual serial ports, one for the double to send the data, and the other one for the application to receive it. This way, the EasyNMEA application acts in the same way as a real application would, so public APIs can be tested in the same manner that they would be used in real applications. The relationships between the different system test components and the sequence of operations are shown in the following diagrams.

@startuml caption System tests components [system_tests.py] <.up. [system_tests.yaml] : loads [system_tests.py] <.up. [expected_results.yaml] : loads package "NMEA Device Double" as nmea_device { [send2serial.py] <.. [easynmea_sentences.nmea] : loads } package "Virtual Serial Ports" as virtual_ports { send_port - [socat] : creates [socat] - recv_port : creates } package "EasyNMEA Application" as easynmea_app { [system_tests] .> [results.yaml] : writes } [send2serial.py] -down-> send_port : writes recv_port -up-> [system_tests] : reads [system_tests.py] --> [send2serial.py] : runs [system_tests.py] --> [socat] : runs [system_tests.py] --> [system_tests] : runs [system_tests.py] <.. [results.yaml] : validates @enduml

@startuml caption System tests sequence diagram control system_tests.py database system_tests.yaml participant socat queue send_port queue recv_port participant send2serial.py database easynmea_sentences.nmea participant system_tests database results.yaml database expected_results.yaml == Arrange phase == [-> system_tests.py: Start test activate system_tests.py system_tests.py <- system_tests.yaml: Load system_tests.py -> socat: Run activate socat system_tests.py -> send2serial.py: Run activate send2serial.py system_tests.py -> system_tests: Run activate system_tests socat -> send_port: Create activate send_port socat -> recv_port: Create activate recv_port send2serial.py <- easynmea_sentences.nmea: Load == Act phase == send2serial.py --> send_port: Write send_port --> socat: Propagate socat --> recv_port: Propagate system_tests <-- recv_port: Read system_tests --> results.yaml: Log ... Write to and read form serial ports until timeout ... == Close phase == system_tests.py --> system_tests: SIGTERM after timeout system_tests --> system_tests.py: Return with exit code deactivate system_tests system_tests.py --> send2serial.py: SIGKILL after timeout deactivate send2serial.py send2serial.py --> system_tests.py: Return with exit code system_tests.py --> socat: SIGKILL after timeout socat -> send_port: Close socat <-- send_port: Closed deactivate send_port socat -> recv_port: Close socat <-- recv_port: Closed deactivate recv_port socat --> system_tests.py: Return deactivate socat == Assert phase == system_tests.py <- results.yaml: Load system_tests.py <- expected_results.yaml: Load system_tests.py -> system_tests.py: Validate results [<- system_tests.py: Report test result deactivate system_tests.py @enduml

  1. gpgga_read_some_and_close: Open a pair of serial ports, send some valid NMEA sentences in one, and read GPGGA data on the other. Then, first close the EasyNMEA and then close the ports. Validate results against expectations.

  2. port_closed_externally: Open a pair of serial ports, send some valid NMEA sentences in one, and read GPGGA data on the other. Then, close the serial ports with the EasyNMEA still opened. The application should detect this an exist gracefully. Validate results against expectations.

  3. stop_sending_data: Open a pair of serial ports, send some valid NMEA sentences in one, and read GPGGA data on the other. Stop sending data before stopping the EasyNMEA. Close the EasyNMEA, then the sending app, and lastly close the ports. Validate results against expectations.

  4. late_sending: Open a pair of serial ports. Then, first start a EasyNMEA, and after 1 second start sending some valid NMEA sentences. Then, close the EasyNMEA before closing the ports. Validate results against expectations.