How do I parse XML with repeating attribute values?

1arryb has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Monks

I have a program which parses the XML feed from the National Vulnerability Database (http://nvd.nist.gov). You can see a sample of the feed at http://nvd.nist.gov/download/nvdcve-2007.xml (warning: don't open this file in a browser unless you want to wait for awhile). The feed is organized by "entry", each entry being the complete report regarding a single software vulnerability. My problem is with the "vuln_soft" tag which lists each vulnerable piece of software, by product "name" and "vendor", and then version "num" and "edition". The structure of this tag is:

<nvd>
  <entry>
    ...
    <vuln_soft>
      <prod name="name1", vendor="vendor1">
        <vers num="1.0" edition=/>
        <vers num="1.1" edition=/>
        ...
      </prod>
      <prod name="name2", vendor="vendor2">
        <vers num="1.0" edition="ee"/>
        <vers num="1.1" edition="ee"/>
        ...
      </prod>
      ...
    </vuln_soft>
  </entry>
<nvd>
[download]

This all works fine except when the name attribute for 2 <prod> tags have the same value. This is unfortunate because it's perfectly legitimate for the same "product" to be distributed by 2 different "vendors".

A real-world example of this is the vulnerability CVE-2007-5333 (from the file, above) which reports vulnerabilities in tomcat from 2 vendors, "apache" and "apache_software_foundation", each with different version lists.

  ...
  <vuln_soft>
    <prod vendor="apache" name="tomcat">
      <vers num="4.1.10" />
      ...
    </prod>
    <prod vendor="apache_software_foundation" name="tomcat">
      <vers num="4.1" />
      ...
    </prod>
  </vuln_soft>
  ...
[download]

XML::Simple won't create a list of tomcat entries, by vendor. Instead, it drops all of the tomcat "products" except the last one parsed.

Arguably this isn't well-formed XML (at least, XML::Simple complains about it when I turn on strict mode), but who am I to be arguing with the Feds? Am I going to have to start messing around with custom handlers in XML::Parser? Would that even work with this dodgy XML, since XML::Simple is already based on XML::Parser?

Thanks

Larry Barnett

Comment on How do I parse XML with repeating attribute values? Select or Download Code

Replies are listed 'Best First'.
Re: How do I parse XML with repeating attribute values? by ikegami (Patriarch) on Dec 12, 2011 at 18:07 UTC
"`name`" isn't a key, so tell XML::Simple it isn't. `KeyAttr => []` [download] Arguably this isn't well-formed XML I don't think so. I don't know of any constraint on the value of fields named "`name`".	[reply] [d/l] [select]
Re: How do I parse XML with repeating attribute values? by ikegami (Patriarch) on Dec 12, 2011 at 18:11 UTC
"`name`" isn't a key, so tell XML::Simple it isn't. `KeyAttr => []` [download] Arguably this isn't well-formed XML I don't think so. I don't know of any constraint on the value of fields named "`name`".	[reply] [d/l] [select]
Re^2: How do I parse XML with repeating attribute values? by 1arryb (Acolyte) on Dec 12, 2011 at 19:07 UTC
ikegami, Thanks, you are quite correct. I was screwed up by the rather too cute default value for KeyAttr. From the XML::Simple perldoc: Note 1: The default value for 'KeyAttr' is 'name', 'key', 'id'. (Note to self: I will read the manual. I will read the manual. I will RTFM ...) Larry	[reply]
Re^3: How do I parse XML with repeating attribute values? by ikegami (Patriarch) on Dec 13, 2011 at 00:18 UTC
I don't blame you. It's a bad default.	[reply]


"be consistent"
	PerlMonks