Behaviour of parsed XML

dalgetty has asked for the wisdom of the Perl Monks concerning the following question:

Dear Brethren,

I know that Perl is not inconsistent, so it must be me.

For years I have been using XML::Simple to parse several RSS feeds, and have 1500 lines of code running nicely, except for the rare occasion when there is only one item in the RSS feed.

In this case the script fails, because $data->{channel}->{item}->[0] does not exist. Since there is only one entry, XML::Simple does not create {item}->[0], but puts the hash table straight into $data->{channel}.

So I adjust the hash table as follows, and I can access the information I need:

$mydata=$data->{channel}->{item};
my $data->{channel}->{item}->[0]=$mydata;
if ($data->{channel}->{item}->[$y]) {
 while (($data->{channel}->{item}->[$y])&&($y>-1)) {
  $keyword=$data->{channel}->{item}->[$y]->{epfl_keywords};
  ...
[download]

The data is then correctly constructed:

$VAR1 = { 'channel' => { 'item' => [ { 'epfl_is_internal' => 'False', 
+'link' => 'http://memento.epfl.ch/event/an-account-of-the-world-s-fir
+st-cubesat/', 'epfl_organizer' => 'eSpace ', 'pubDate' => 'Mon, 16 Ma
+r 2020 14:00:00 +0100', 'description' => "Incl
...
[download]

However, this code needs to run when there are several items in the RSS feed too, so I only want to apply the above operation in cases of one item. In order to test for this I use the following code:

if (exists($data->{channel}->{title})) {
 $mydata=$data->{channel}->{item};
 my $data->{channel}->{item}->[0]=$mydata;
 print "Only one item is present"; 
}
if ($data->{channel}->{item}->[$y]) {
 while (($data->{channel}->{item}->[$y])&&($y>-1)) {
  $keyword=$data->{channel}->{item}->[$y]->{epfl_keywords};
  ...
[download]

"title" is one of many keys that always exists in the RSS feed entries. If it exists directly within "channel" that means that there is only one RSS entry in the feed, and the message prints out "Only one item is present". So far, so good.

However, I then get an error: "Not an ARRAY reference" for the second "if" statement in line 6, as if the restructuring had not happened.

This seemed strange to me, since the single entry case had clearly been identified correctly. So I tried the following:

if (1) {
 $mydata=$data->{channel}->{item};
 my $data->{channel}->{item}->[0]=$mydata;
}
if ($data->{channel}->{item}->[$y]) {
 while (($data->{channel}->{item}->[$y])&&($y>-1)) {
  $keyword=$data->{channel}->{item}->[$y]->{epfl_keywords};
  ...
[download]

I fully expected this code to run smoothly, like the first attempt did. Of course, I would never have attempted to make Perl look inconsistent. But I get the "not an array" error again.

All I am doing is testing for something, not changing anything. But just making an if(1) statement is enough to stop my code working correctly. What is even more confusing is that a Dumper print of the data shows that the data is correctly structured, as in the first statement, whether I apply the if statement or not.

Can any of you please tell me where my inconsistency lies? Thanks to all

Comment on Behaviour of parsed XML Select or Download Code

Replies are listed 'Best First'.
Re: Behaviour of parsed XML by haukex (Archbishop) on Feb 26, 2020 at 11:22 UTC
An SSCCE would be helpful here; for example we don't know if you're using the `ForceArray` option. In any case, XML::Simple is extremely brittle (see the warning at the top of its documentation) and I would very strongly recommend trying to move away from it. I've shown how to use XML::Rules as a replacement several times, see this node and the links therein.	[reply] [d/l]
Re: Behaviour of parsed XML by cavac (Parson) on Feb 26, 2020 at 14:05 UTC
Try: `my $xml = XMLin($filename, ForceArray => ['item'];` [download] `perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'`	[reply] [d/l] [select]
Re^2: Behaviour of parsed XML by dalgetty (Acolyte) on Feb 26, 2020 at 16:17 UTC
Wow. cavac That just worked. Like a turnaround jumpshot. Thanks to everybody for your help, it's all good now...	[reply]
Re^3: Behaviour of parsed XML by tobyink (Canon) on Feb 26, 2020 at 18:18 UTC
It might solve this particular issue, but in general moving away from XML::Simple will serve you better long term. The module is conceptually broken. It treats XML as being basically weird-looking JSON. But XML's data model is entirely different than JSON's. Not going to get into a debate about which data model is better, but treating them the same is like writing flight-planning software that will only generate flight routes that follow highways. Somebody else recommended XML::Rules, which is probably the easiest thing to transition to from XML::Simple. It still allows you to treat the XML like glorified JSON, but requires you to give it rules about how to translate between the models, with some sane defaults. The other thing I'd recommend looking at is XML::LibXML, which provides a full DOM API for XML, which will be pretty familiar if you've done any client-side Javascript programming. toby döt ink	[reply]
Re^4: Behaviour of parsed XML by cavac (Parson) on Feb 28, 2020 at 14:35 UTC
Re^5: Behaviour of parsed XML by hippo (Bishop) on Feb 28, 2020 at 14:49 UTC
Re^5: Behaviour of parsed XML by Jenda (Abbot) on Feb 28, 2020 at 17:12 UTC
Variable scope issue- effect of "my" on hash ref keys, values by parv (Parson) on Feb 26, 2020 at 11:29 UTC
I hate the software that does not keep the return value consistent. Short of asking a parser for an array reference (somehow) regardless of number of items present, however, I do not see a parser to always return an array reference in the situation described. With that out of the way ... Your problem seems to be of variable scope. Why do you have "`my`" in ... `if (exists($data->{channel}->{title})) { ... my $data->{channel}->{item}->[0]=$mydata; ... }` [download] ...? After the end of the if-block, `$data->{channel}->{item}->[0]` ceases to exist due to my operator. ~~Read more... I had forgotten my Perl (863 Bytes)~~ After tobyink & haukex had set me straight, please ignore the above code; try this ... `use strict; use warnings; use Data::Dumper; my $x; { my $x->{a}->[0] = "no you don't!"; } { $x->{b}->[0] = "now you see"; } print Dumper $x; __END__ $VAR1 = { 'b' => [ 'now you see' ] };` [download]	[reply] [d/l] [select]
Re: Variable scope issue- effect of "my" on hash ref keys, values by tobyink (Canon) on Feb 26, 2020 at 13:16 UTC
Why would the "extra fun" bit be in the output? You declared two separate `$x` variables and they're each references to different hashes. If you put data into one hash, it shouldn't appear in the other one. The only arguable improvement which could be made to Perl with regard to this, is that `my $x->{a}->[0]` is kinda weird and might be worth warning about. Really that's just a precedence thing though. `my` is higher precedence than the deref operator, so it just means `(my $x)->{a}->[0]`. toby döt ink	[reply] [d/l] [select]
Re^2: Variable scope issue- effect of "my" on hash ref keys, values by parv (Parson) on Feb 26, 2020 at 13:24 UTC
Ha! Right you are. I had missed the fact a spanking new variable was created inside the block (when assigning to the hash reference keys). No "extra fun" for me, it's obvious now. I was mistakenly thinking that `my` operator was affecting the existence of keys, values (in that `{a}` element was autovivified and only `[0]` element was localized to the block) but not the `$x` created earlier outside the block. Much thanks to you both for the clue bat.	[reply] [d/l] [select]
Re: Variable scope issue- effect of "my" on hash ref keys, values by haukex (Archbishop) on Feb 26, 2020 at 13:16 UTC
I am disturbed by the fact that "extra fun" does not make into the Dumper output I'm pretty sure that's because `my $x->{a}->[0] = "no you don't!";` is mostly equivalent to `my $x; $x->{a}->[0] = "no you don't!";`. From Private Variables via my(): All listed elements must be legal lvalues. ... The `my` is simply a modifier on something you might assign to.	[reply] [d/l] [select]
Re: Variable scope issue- effect of "my" on hash ref keys, values by Veltro (Hermit) on Feb 26, 2020 at 13:23 UTC
In perl 5.30.1, I am disturbed by the fact that "extra fun" does not make into the Dumper output (but not disturbed enough to file a problem report myself). Not 100% sure what you mean, but to me that makes sense. You still have `my $x->...` declared in the scope of the first block the next statement just adds to that locally scoped var inside that same block.	[reply] [d/l]


Syntactic Confectionery Delight
	PerlMonks