Re: bracket processing

Thank you Monks, great points all!

As suggested by @Ken and @AnonymousMonk, let me elaborate the problem. The overall goal is to remove all types of brackets from 'noisy' text (e.g html content/tweets etc.), thereby 'sanitize' text. The brackets may appear in text in any number and in any form (edge cases), the idea is to remove content from within all non-overapping, longest-extending, balanced brackets regardless of their types. Strings that have unbalanced brackets can be ignored. Flanking characters of brackets may be among (\s or \. or \; or \: or \,).

The below script from Ken's & AnonymousMonk's suggestions works well if there is just one 'big' bracket, as:

Program:

####program.pl
#!/usr/bin/perl
use strict;
use warnings;
use Text::Balanced 'extract_bracketed';

my $delim = '([{<';
my $prefix = qr{[^\Q$delim\E]*};
my $string = 'The use of parentheses (indicates that the (writer [cons
+idered] the {information}) less <important—almost> an afterthought).'
+;
my @parts = extract_bracketed($string, $delim, $prefix);
$parts[2]=~s/\s*$//;
print WF1 "pattern:\'$parts[0]\'\n";
print WF1 "rightside of pattern:\'$parts[1]\'\n";
print WF1 "leftside of pattern:\'$parts[2]\'\n";
[download]

Output:

pattern:'(indicates that the (writer [considered] the {information}) l
+ess <important—almost> an afterthought)'
rightside of pattern:'.'
leftside of pattern:'The use of parentheses'
[download]

However, when another non-overlapping bracket appears in the string (e.g. '(use {of})') as shown below, the above script removes just one, as shown below:

$string = 'The (use {of}) parentheses (indicates that the (writer [con
+sidered] the {information}) less <important—almost> an afterthought).
+';

Output:
pattern:'(use {of})'
rightside of pattern:' parentheses (indicates that the (writer [consid
+ered] the {information}) less <important—almost> an afterthought).'
leftside of pattern:'The'
[download]

The desired output though should have the second bracket also removed, something along the lines:

Desired output:'The parentheses.';
Pattern1removed:'(use {of})'
Pattern2removed:'(indicates that the (writer [considered] the {information}) less <important—almost> an afterthought)'

How can such cases be addressed?

Thanks again your help!

Comment on Re: bracket processing Select or Download Code

Replies are listed 'Best First'.
Re^2: bracket processing by kcott (Archbishop) on Apr 01, 2020 at 02:31 UTC
You could just use a loop, dealing with each bracketed part that is encountered. Here's a basic technique: #!/usr/bin/env perl use strict; use warnings; use constant { EXTRACTED => 0, SUFFIX => 1, PREFIX => 2, }; use Text::Balanced 'extract_bracketed'; my $delim = '([{<'; my $prefix = qr{[^$delim]}; my $string = 'The (use {of}) parentheses (indicates that the (writer [ +considered] the {information}) less <important—almost> an afterthough +t).'; my ($current, $wanted) = ($string, ''); while (1) { my @parts = extract_bracketed($current, $delim, $prefix); if (defined $parts[PREFIX]) { my ($trimmed_start) = $parts[PREFIX] =~ /^(.?)\s$/; $wanted .= $trimmed_start; } if (defined $parts[EXTRACTED]) { $current = $parts[SUFFIX]; } else { $wanted .= $parts[SUFFIX]; last; } } print "$wanted\n"; [download] Output: `The parentheses.` [download] I'll leave you to integrate that technique* into your actual production code. [The lack of sample data — as requested by myself and expanded upon by AnomalousMonk — was disappointing.] — Ken	[reply] [d/l] [select]
Re^3: bracket processing by rajaman (Sexton) on Apr 02, 2020 at 02:16 UTC
Excellent, thanks much Ken. That helps! Sorry I missed samples, attaching a few here. I am starting to learn text analysis (e.g. sentiment analysis on reviews/opinion). Here is the link to the full dataset: http://www.cs.jhu.edu/~mdredze/datasets/sentiment/ . Many are also available on google. Few samples are shown below. Seems text processing is far more complex than I realized. Lots of variations/edge cases. Just came across cases where brackets can't be removed as it makes the text pretty ackward, something like: 'Company is good because of: (1) quick service, (2) product range..'. #samples The largest of these (4 quart{16 cups}) will hold all but a little bit + of a bag of flour. I like the look of the stainless steel and the fa +ct of the seal around the lid. I {mistakenly} did not recieve my order (it was a different order i wa +s expecting), and they rushed me another collar. The rougher the machine the more wear on the fabric and more pilling, +lint, etc.... I would highly recommend these sheets for the price an +d the great quality for the price. [I would rate them 5 stars if the +y were heavier and soft as cashmere, but for the money definitely a g +ood buy.] [download]	[reply] [d/l]
Re^4: bracket processing by AnomalousMonk (Archbishop) on Apr 02, 2020 at 04:09 UTC
Please let me suggest once again a reading of How to ask better questions using Test::More and sample data. What I and some other monks would like to see is something like: my @Tests = ( [ 'The largest (4 quart{16 cups}) will hold most of a bag of flour +.', # input string 'The largest will hold most of a bag of flour.', + # expected output ], [ 'I {mistakenly} did not recieve my order (it was a different ord +er), and they rushed me another.', 'I did not recieve my order, and they rushed me another.', ], [ 'Company is good because of: (1) quick service, (2) product rang +e.', 'Company is good because of: (1) quick service, (2) product rang +e.', 'parenthesized ordinals untouched', # optional comment ], [ '...', '...', ], # another test case ... # more test cases ); [download] This can integrate easily into a Test::More testing framework. Individual test cases should be as short as possible and should always be paired with exact expected output. Degenerate and trivial cases, e.g., `'' ' ' '()' ' ()' '() ' 'x' 'foo' etc, etc...` should also be given. (Update: If exceptions may be thrown, a test case or cases should ultimately be written for each exception; see Test::Exception.) Note that this is work you will have to do anyway because you must always* build a testing framework for any serious project you undertake!* What I, for one, do not like to see is an invitation to do a lot of work to generate test case input for which I must also guess the correct output; that's your job. Give a man a fish: `<%-{-{-{-<`	[reply] [d/l] [select]
Re^5: bracket processing by rajaman (Sexton) on Apr 05, 2020 at 21:00 UTC