[Buildroot] [PATCH 08/17] support/scripts/cpe-report: new script

[Heiko Thiery]

Hi Gregory, Hi Matt,

Am Di., 6. Okt. 2020 um 15:44 Uhr schrieb Gregory CLEMENT
<gregory.clement at bootlin.com>:
>
> From: Matt Weber <matthew.weber at rockwellcollins.com>
>
> The script supports looking up all the CPEs provided in a
> make cpe-info csv file export from a target Buildroot build.
> It checks the current version and suggests a CPE needs update
> or possibly an initial submission is required to NIST.

Is there a way to create this kind of list/output also for all
packages in buildroot and not only the one that is generated by a
configuration?

> Adds option to allow alternate locations for the dictionary
> URL and caching of a processed dictionary to speed up execution.
>
> Outputs a cpe/ folder with propsed xml generated from the
> dictionary contents to propose updated versions to NIST.
>
> For missing CPE matches, a cpe-report-missing.txt is created
> by the script that can be used later to manually create proposed
> new NIST dictionary entries.
>
> Ref: NIST has a group email (cpe_dictionary at nist.gov) used to
> recieve these version update and new entry xml files.  They do
> process the XML and provide feedback. In some cases they will
> propose back something different where the vendor or version is
> slightly different.
>
> Limitations
>  - Currently any use of non-number version identifiers isn't
>    supported by NIST as they use ranges to determine impact
>    of a CVE
>  - Any Linux version from a non-upstream is also not supported
>    without manually adjusting the information as the custom
>    kernel will more then likely not match the upstream version
>    used in the dictionary
>
> Signed-off-by: Matt Weber <matthew.weber at rockwellcollins.com>
> ---
>  support/scripts/cpe-report | 70 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 70 insertions(+)
>  create mode 100755 support/scripts/cpe-report
>
> diff --git a/support/scripts/cpe-report b/support/scripts/cpe-report
> new file mode 100755
> index 0000000000..7242a372b2
> --- /dev/null
> +++ b/support/scripts/cpe-report
> @@ -0,0 +1,70 @@
> +#!/usr/bin/env python

Shouldn't we use python3 here?

> +
> +import argparse
> +import sys
> +import csv
> +from cpedb import CPEDB
> +
> +CPE_XML_URL = "https://static.nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz"
> +
> +
> +def get_target_cpe_report(cpe_report_file, cpedb):
> +    report_cpe_exact_match = ""
> +    report_cpe_needing_update = ""
> +    report_cpe_needing_update_list = ""
> +    report_cpe_missing = ""
> +
> +    print("CPE: Checking for matches...")
> +    try:
> +        with open(cpe_report_file) as cpe_file:
> +            cpe_list = csv.reader(cpe_file)
> +            next(cpe_list)  # make cpe-info has a one line header
> +            for cpe in cpe_list:
> +                result = cpedb.find(cpe[0])
> +                if not result:
> +                    result = cpedb.find_partial(cpedb.get_cpe_no_version(cpe[0]))
> +                    if not result:
> +                        report_cpe_missing += cpe[0] + "," + cpe[1] + "," + cpe[3] + "\n"
> +                    else:
> +                        latest_version = cpedb.find_partial_latest_version(cpedb.get_cpe_no_version(cpe[0]))
> +                        report_cpe_needing_update += cpe[0] + ", Latest Version Guess from Dict[" + latest_version + "]\n"
> +                        report_cpe_needing_update_list += cpe[0] + "\n"
> +                else:
> +                    report_cpe_exact_match += cpe[0] + "\n"
> +    except (OSError, IOError) as e:
> +        print("CPE: report csv file (%s): %s" % (e.errno, e.strerror))
> +        sys.exit(1)
> +
> +    print("CPE: Found but may REQUIRE an UPDATE:\n" + report_cpe_needing_update)
> +    print("CPE: Not found:\n" + report_cpe_missing)
> +
> +    fp = open('cpe-report-missing.txt', 'w+')
> +    fp.write(report_cpe_missing)
> +    fp.close()
> +
> +    for cpe in report_cpe_needing_update_list.splitlines():
> +        cpedb.update(cpe)
> +    print("XML Generation Complete of NIST update files, see ./cpe/*")
> +
> +
> +def parse_args():
> +    parser = argparse.ArgumentParser()
> +    parser.add_argument('-c', dest='cpe_report', action='store', required=True,
> +                        help='CPE Report generated by make cpe-info (csv format)')
> +    parser.add_argument('-u', dest='url', action='store', required=False,
> +                        help='(optional)URL to the NIST dict (official-cpe-dictionary_v2.3.xml.gz)')
> +    return parser.parse_args()
> +
> +
> +def __main__():
> +    args = parse_args()
> +    cpedb = CPEDB()
> +    url = CPE_XML_URL
> +    if args.url:
> +        url = args.url
> +    cpedb.get_xml_dict(url)
> +    print("Performing Target CPE Report Analysis...")
> +    get_target_cpe_report(args.cpe_report, cpedb)
> +
> +
> +__main__()

-- 
Heiko