Skip to content

Add tests using real files #107

@terriko

Description

@terriko

To make the basic test suite run quickly, we use "faked" binary files to test the CVE mappings. However, we want to be able to test real files to test that the signatures work on real-world data.

In #99, I've added a _file_test function (to match the existing _binary_test) that takes a url, and package name and a version, and downloads the file, runs the scanner against it, and makes sure it is the file that you've specified. But we need more tests!

  • Existing tests are in test/
  • You can see the scanner tests in 'tests/test_scanner.py'
  • To add a new test, find an appropriate publicly available file (linux distribution packages and public releases of the packages itself are ideal)
  • Make sure to hide it behind the LONG_TESTS flag so we aren't doing huge number of downloads for every test suite run
    @unittest.skipUnless(os.getenv('LONG_TESTS') == '1', 'Skipping long tests')
    def test_rpm_curl_7_32_0(self):
        """
        test to see if we detect a real copy of curl 7.32.0
        """
        self._file_test(
            'https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Everything/x86_64/os/Packages/c/',
            'curl-7.32.0-3.fc20.x86_64.rpm',
            'curl',
            '7.32.0')

I'd like to have at least one test for every checker, and it would be nice to have some different sources for each as well. For example, for packages available in common Linux distributions, we might want to have one from fedora, one from debian, and one direct from upstream to show that we detect all those versions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomershacktoberfestgood issue for hacktoberfest participation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions