paste.chapril.org-privatebin/tst
2017-10-08 16:42:11 +02:00
..
Data Merge branch 'master' into js-unit-testing 2017-03-25 09:06:04 +01:00
Persistence Use better random number generator #29 2016-08-10 23:15:06 +02:00
Bootstrap.php updating unit test in preparation for planned file name change, currently failing 2017-10-08 16:25:11 +02:00
ConfigurationTest.php implemented INI cenversion functionality 2017-10-08 16:42:11 +02:00
ConfigurationTestGenerator.php Fix style of operator spaces 2016-10-31 11:32:12 +01:00
FilterTest.php bumping required PHP to 5.4, removing unneccessary code, resolves #186 2017-03-05 11:22:24 +01:00
I18nTest.php added missing/removed translation IDs found using improved unit test (#201) 2017-03-25 13:19:11 +01:00
JsonApiTest.php changes the file extension to php and adds a small one-liner to stop PHP from presenting the file to any website visitor 2017-10-08 16:25:48 +02:00
ModelTest.php applying styleCI recommendations 2017-03-24 21:35:50 +01:00
phpunit.xml starting to work on JSVerify & Mocha based unit tests for our JS code base 2017-01-29 14:31:44 +01:00
PrivateBinTest.php added instantburnafterreading option to address #174 2017-04-11 17:23:26 +02:00
PrivateBinWithDbTest.php applying patch based on StyleCI ruleset 2016-10-29 10:24:08 +02:00
README.md Merge branch 'master' into js-unit-testing 2017-05-22 22:38:49 +02:00
RequestTest.php removing duplicate code, cleanup of temporary test files 2017-03-24 23:42:11 +01:00
SjclTest.php updated .htaccess format, refactored .htaccess creation logic and improving code coverage, fixes #194 2017-03-24 21:30:08 +01:00
ViewTest.php fixing page template, removing error messages when markdown or source are disabled in configuration, re-removing unnecessary spans 2017-03-12 14:16:08 +01:00
Vizhash16x16Test.php applying patch based on StyleCI ruleset 2016-10-29 10:24:08 +02:00

Running PHP unit tests

In order to run these tests, you will need to install the following packages and their dependencies:

  • phpunit
  • php-gd
  • php-sqlite3
  • php-xdebug (for code coverage reports)

Example for Debian and Ubuntu:

$ sudo apt install phpunit php-gd php-sqlite3 php-xdebug

To run the tests, change into the tst directory and run phpunit:

$ cd PrivateBin/tst
$ phpunit

Additionally there is the ConfigurationTestGenerator. Based on the configurations defined in its constructor, it generates the unit test file tst/ConfigurationCombinationsTest.php, containing all possible combinations of these configurations and tests for (most of the) valid combinations. Some of combinations can't be tested with this method, i.e. a valid option combined with an invalid one. Other very specific test cases (i.e. to trigger multiple errors) are covered in tst/PrivateBinTest.php. Here is how to generate the configuration test and run it:

$ cd PrivateBin/tst
$ php ConfigurationTestGenerator.php
$ phpunit ConfigurationCombinationsTest.php

Note that it can take an hour or longer to run the several thousand tests.

Running JavaScript unit tests

In order to run these tests, you will need to install the following packages and its dependencies:

  • npm

Then you can use the node package manager to install the latest stable release of mocha and istanbul (for code coverage reports) globally and jsVerify, jsdom and jsdom-global locally:

$ npm install -g mocha istanbul
$ cd PrivateBin/js
$ npm install jsverify jsdom@9 jsdom-global@2

Example for Debian and Ubuntu, including steps to allow the current user to install node modules globally:

$ sudo apt install npm
$ sudo mkdir /usr/local/lib/node_modules
$ sudo chown -R $(whoami) $(npm config get prefix)/{lib/node_modules,bin,share}
$ ln -s /usr/bin/nodejs /usr/local/bin/node
$ npm install -g mocha istanbul
$ cd PrivateBin/js
$ npm install jsverify jsdom@9 jsdom-global@2

Note: If you use a distribution that provides nodeJS >= 6, then you can install the latest jsdom and jsdom-global packages and don't need to use @9 and @2.

To run the tests, just change into the js directory and run istanbul:

$ cd PrivateBin/js
$ istanbul cover _mocha

Property based unit testing

In the JavaScript unit tests we use the JSVerify library to leverage property based unit testing. Instead of artificially creating specific test cases to cover all relevant paths of the tested code (with the generated coverage reports providing means to check the tested paths), property based testing allows us to describe the patterns of data that are valid input.

With each run of the tests, for each jsc.property 100 random inputs are generated and tested. For example we tell the test to generate random strings, which will include empty strings, numeric strings, long strings, unicode sequences, etc. This is great for finding corner cases that one might not think of when explicitly writing one test case at a time.

There is another benefit, too: When an error is found, JSVerify will try to find the smallest, still failing test case for you and print this out including the associated random number generator (RNG) state, so you can reproduce it easily:

[...]

  30 passing (3s)
  1 failing

  1) Helper getCookie returns the requested cookie:
     Error: Failed after 30 tests and 11 shrinks. rngState: 88caf85079d32e416b; Counterexample: ["{", "9", "9", "YD8%fT"]; [" ", "_|K:"]; 

[...]

Of course it may just be that you need to adjust a test case if the random pattern generated is ambiguous. In the above example the cookie string would contain two identical keys "9", something that may not be valid, but that our code could encounter and needs to be able to handle.

After you adjusted the code of the library or the test you can rerun the test with the same RNG state as follows:

$ istanbul cover _mocha -- test.js --jsverifyRngState 88caf85079d32e416b