Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

[OT] MySQL recalibrating a sort-index

by LanX (Saint)
on Feb 15, 2018 at 19:00 UTC ( [id://1209251]=perlquestion: print w/replies, xml ) Need Help??

LanX has asked for the wisdom of the Perl Monks concerning the following question:

Hi

There is an actual Perl context, I'm representing (kind of) a HoAoHoA tree structure in an SQL-table (so it's not totally off-topic).

The array part is using a field f_sort to represent the order.

For instance: (the parent_id=2 identifies a particular array inside the tree)

f_node_id f_parent_id f_name f_sort
142 2 B06 1
143 2 B2L 2
144 2 B2M 3
145 2 B2N 4
146 2 B2O 5

Now I have the requirement to delete , insert and move ° multiple elements in this "array", which of course involves renumbering f_sort .

I wanted to be clever and avoid thinking about all involved edge cases, and changed the type of f_sort to float, such that I can just move new elements in between two elements as fractions. (the monastery is using a similar concept for sorting in user settings)

f_node_id f_parent_id f_name f_sort
142 2 B06 1
143 2 B2L 2
394 2 *Inserted* 2.5
144 2 B2M 3
145 2 B2N 4
146 2 B2O 5

And after each operation I would just run one SQL statement to "refresh" the f_sort index to integers starting from 1.

SET @sort = 0; UPDATE t_tree AS T SET f_sort = (@sort := @sort +1) WHERE T.f_parent_id = 2 ORDER BY f_sort;

Now this fails since (f_parent_id, f_sort) are set to be unique, i.e. numbering the inserted element 394 to f_sort :=3 will clash with the already available element 144 with f_sort=3 (and so on)

Of course I could start here sorting from the end, but this contradicts the intention to avoid creating code for each case.

A workaround is to multiply each f_sort with -1 before re-sorting, because new indices can't clash with negative numbers.

But this feels quite clumsy.

I could achieve the same effect with an extra boolean column f_is_currently_sorting and add it to the "unique index" (it would reflect the "negativity" of -1) ... at least the sort order would stay correct in every phase. But still.

Any better ideas for that approach?

Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Wikisyntax for the Monastery

°) move in the sense of cut&paste or copy&paste of multiple elements, be from the same or from another array

update

I was asked

> I'm not clear on what you're wanting to do. Can you add a bit of information on what the sort criteria are?

ORDER BY f_sort WHERE f_parent_id = ?

f_node_id is the primary key f_parent_id is the node_id of an upper structure f_name is the key of a hash if f_sort= NULL, otherwise it's just an optional "name" of a array element +* f_sort is the array index and NULL for hashes

so yes the sort criteria is actually

ORDER BY f_sort,f_name WHERE f_parent_id = ?

but the hash logic is not relevant for this question.

*) think sorted hashes, or representing JSON, where some RFC require that the order of keys matters

Replies are listed 'Best First'.
Re: [OT] MySQL recalibrating a sort-index
by erix (Prior) on Feb 16, 2018 at 11:50 UTC

    UPDATE: I added an CONSTRAINT DEFERRABLE to the create table, which renders LanX comment obsolete. Sorry about that./UPDATE

    This is not for Oracle MySQL, I think (I didn't find the necessary functionality). This is just for fun, therefore Postgres :P

    This won't work for you but you expressed some interest (or what I took to be interest) in the CB so here goes:

    Below is the (p)sql-inside-bash that I cobbled together; I inserted the database-output in the script ( /* commented out */ ) for better reading.

    #!/bin/sh t=ctetree echo " \set insert_location 2.5 begin transaction; drop table if exists $t ; create table $t ( f_node_id int primary key , f_parent_id int , f_name text , f_sort float , constraint tree_parent_sort_uniq_idx unique (f_parent_id, f_sort) d +eferrable ); insert into $t values (142, 2, 'B06', 1) , (143, 2, 'B2L', 2) , (144, 2, 'B2M', 3) , (145, 2, 'B2N', 4) , (146, 2, 'B2O', 5) ; \echo -- base data: table $t; /* -- base data: f_node_id | f_parent_id | f_name | f_sort -----------+-------------+--------+-------- 142 | 2 | B06 | 1 143 | 2 | B2L | 2 144 | 2 | B2M | 3 145 | 2 | B2N | 4 146 | 2 | B2O | 5 (5 rows) */ insert into $t values (394, 2, '*Inserted 1st*', :insert_location); update $t set f_sort = floor(f_sort)+1 where f_sort >= :insert_locatio +n; /* THIS BETTER REPLACED BY... see my later posts downstream */ \echo -- new begin state, after 1st insert and renumbered: table $t order by f_sort; /* -- new begin state, after 1st insert and renumbered: f_node_id | f_parent_id | f_name | f_sort -----------+-------------+----------------+-------- 142 | 2 | B06 | 1 143 | 2 | B2L | 2 394 | 2 | *Inserted 1st* | 3 144 | 2 | B2M | 4 145 | 2 | B2N | 5 146 | 2 | B2O | 6 (6 rows) */ with inserted as ( insert into $t values (395, 2, '*Inserted 2nd*', :insert_location) returning f_sort ) update $t set f_sort = floor(f_sort)+1 where f_sort > (select f_sort +from inserted); \echo -- after 2nd insert table $t order by f_sort; /* -- after 2nd insert f_node_id | f_parent_id | f_name | f_sort -----------+-------------+----------------+-------- 142 | 2 | B06 | 1 143 | 2 | B2L | 2 395 | 2 | *Inserted 2nd* | 2.5 394 | 2 | *Inserted 1st* | 4 144 | 2 | B2M | 5 145 | 2 | B2N | 6 146 | 2 | B2O | 7 (7 rows) */ update $t set f_sort = floor(f_sort)+1 where f_sort <> floor(f_sort); \echo -- after 2nd insert - renumbered table $t order by f_sort; /* -- after 2nd insert - renumbered f_node_id | f_parent_id | f_name | f_sort -----------+-------------+----------------+-------- 142 | 2 | B06 | 1 143 | 2 | B2L | 2 395 | 2 | *Inserted 2nd* | 3 394 | 2 | *Inserted 1st* | 4 144 | 2 | B2M | 5 145 | 2 | B2N | 6 146 | 2 | B2O | 7 (7 rows) */ -- -- remove rows 2 and 3, and insert them after the row where f_sort +is 5. -- \set location_to_insert 5 \echo -- initial: table $t order by f_sort; /* -- initial: f_node_id | f_parent_id | f_name | f_sort -----------+-------------+----------------+-------- 142 | 2 | B06 | 1 143 | 2 | B2L | 2 395 | 2 | *Inserted 2nd* | 3 394 | 2 | *Inserted 1st* | 4 144 | 2 | B2M | 5 145 | 2 | B2N | 6 146 | 2 | B2O | 7 (7 rows) */ create temp table _t_$t ( like $t including all ); \echo -- intermediary 1, empty temp table (to receive deleted rows): table _t_$t order by f_sort; /* -- intermediary 1, empty temp table (to receive deleted rows): f_node_id | f_parent_id | f_name | f_sort -----------+-------------+--------+-------- (0 rows) */ \echo -- with deleted -> d. rows inserted into tmp table with deleted as ( delete from $t where f_sort in (2,3) returning * ) insert into _t_$t select * from deleted; \echo -- intermediary 2 (filled with the deleted rows): table _t_$t order by f_sort; /* -- intermediary 2 (filled with the deleted rows): f_node_id | f_parent_id | f_name | f_sort -----------+-------------+----------------+-------- 143 | 2 | B2L | 2 395 | 2 | *Inserted 2nd* | 3 (2 rows) */ \echo -- with updated -> rows inserted in main table with updated as ( update _t_$t set f_sort = :location_to_insert + f_sort / 1000 returning * ) insert into $t table updated; create temp table _t as select *, row_number() over () as new_number from (select * from $t order by f_sort) f order by new_number ; \echo -- semi-ready: table _t order by f_sort; /* -- semi-ready: f_node_id | f_parent_id | f_name | f_sort | new_number -----------+-------------+----------------+--------+------------ 142 | 2 | B06 | 1 | 1 394 | 2 | *Inserted 1st* | 4 | 2 144 | 2 | B2M | 5 | 3 143 | 2 | B2L | 5.002 | 4 395 | 2 | *Inserted 2nd* | 5.003 | 5 145 | 2 | B2N | 6 | 6 146 | 2 | B2O | 7 | 7 (7 rows) */ update $t t set f_sort = _t.new_number from _t where _t.f_node_id = t.f_node_id --> join and _t.new_number <> t.f_sort --> avoid writes ; \echo -- ready: table $t order by f_sort; /* -- ready: f_node_id | f_parent_id | f_name | f_sort -----------+-------------+----------------+-------- 142 | 2 | B06 | 1 394 | 2 | *Inserted 1st* | 2 144 | 2 | B2M | 3 143 | 2 | B2L | 4 395 | 2 | *Inserted 2nd* | 5 145 | 2 | B2N | 6 146 | 2 | B2O | 7 (7 rows) */ rollback; -- clean up " | psql -Xqa | less -iSR

    UPDATE: slightly tweaked UPDATEs

      > This is not for Oracle MySQL, I think (I didn't find the necessary functionality).

      MariaDB to be precise. (Sorry, I adapted to the sloppy wording of my department.)

      > This is just for fun, therefore Postgres :P

      Oh, I expected nothing less - than excellence - from you! :P

      I'll translate it to a "minor form of SQL", have a deeper look into it, and give feedback.

      (Not sure if the floor approach works for multiple changes.)

      Thanks! :)

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Wikisyntax for the Monastery

        Yeah, the other way is more general: create, then join to a temporary table. (that floor() was just a first try, I should have removed it)

        insert into $t values (394, 2, '*Inserted 1st*', :insert_location); -- update $t set f_sort = floor(f_sort)+1 where f_sort >= :insert_loca +tion; -- meh -- instead: -- create temporary table create temp table _t as select f_node_id, f_sort, row_number() over () as new_number from ( select f_node_id, f_sort from $t order by f_sort ) as f order by new_number ; -- update table update $t t set f_sort = _t.new_number from _t where _t.f_node_id = t.f_node_id --> join expression and _t.new_number <> t.f_sort --> avoid unnecessary writes ; drop table _t; --> remove temp table

        It's verbose-ugly but it works without repeating the values.

Re: [OT] MySQL recalibrating a sort-index
by choroba (Cardinal) on Feb 16, 2018 at 08:08 UTC
    What do you mean by "creating code for each case"?
    SELECT 1 + COUNT(1) INTO @sort FROM t_tree AS t WHERE t.f_parent_id = 2 LIMIT 1; UPDATE t_tree AS t SET f_sort = (@sort := @sort - 1) WHERE T.f_parent_id = 2 ORDER BY f_sort DESC;

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
      > What do you mean by "creating code for each case"?

      • Re-numbering from the end covers only the insert case.
      • If you delete elements you need to renumber from the start.
      • If you move elements you need to delete and insert...

        (Move is even trickier because in which state are the elements in between? And what happens if an error occurs then?)

      • There are even two kinds of "move", moving inside the same array and moving from another array.

        (But I think this is easy to handle, just change the parent_id in the second case)

      • Finally there is the copy case, where elements must be (deeply) cloned before inserting. (But this problem exists in all models)

      Hope this is clearer without needing to produce examples.

      My approach allows to stack multiple such operations and just to renumber at the end.

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Wikisyntax for the Monastery

Re: [OT] MySQL recalibrating a sort-index (replace)
by LanX (Saint) on Feb 17, 2018 at 00:23 UTC
    Hmm .. combining a temporary TABLE with REPLACE seems to do the trick.

    DROP TABLE IF EXISTS t_tree_insert, _t; CREATE TABLE `t_tree_insert` ( `f_node_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `f_parent_id` INT(10) UNSIGNED NULL DEFAULT '0', `f_name` VARCHAR(200) NOT NULL, `f_sort` FLOAT NULL DEFAULT NULL, PRIMARY KEY (`f_node_id`), UNIQUE INDEX `sort` (`f_parent_id`, `f_sort`), INDEX `f_parent_id` (`f_parent_id`), FULLTEXT INDEX `f_name` (`f_name`) ) COMMENT='test tree move operations' COLLATE='utf8_general_ci' ENGINE=InnoDB AUTO_INCREMENT=0 ; insert into t_tree_insert values (130, 1, 'other array', 1) , (142, 2, 'B06', 1) , (143, 2, 'B2L', 2) , (144, 2, 'B2M', 3) , (200, 2, 'INSERT', 3.5) , (145, 2, 'B2N', 4) , (146, 2, 'B2O', 5) , (150, 3, 'other array', 1) ; select * from t_tree_insert order by f_parent_id, f_sort; set @sort=0; create temporary table _t as select f_node_id, f_parent_id, f_name, (@sort:=@sort+1) as f_sort from t_tree_insert as T where f_parent_id = 2 order by T.f_sort ; select * from _t; replace into t_tree_insert (f_node_id, f_parent_id, f_name, f_sort) select * from _t ; DROP TABLE _t; select * from t_tree_insert order by f_parent_id, f_sort;

    t_tree_insert
    130 1 other array 1
    142 2 B06 1
    143 2 B2L 2
    144 2 B2M 3
    200 2 INSERT 3,5
    145 2 B2N 4
    146 2 B2O 5
    150 3 other array 1

    _t

    142 2 B06 1
    143 2 B2L 2
    144 2 B2M 3
    200 2 INSERT 4
    145 2 B2N 5
    146 2 B2O 6

    t_tree_insert

    130 1 other array 1
    142 2 B06 1
    143 2 B2L 2
    144 2 B2M 3
    200 2 INSERT 4
    145 2 B2N 5
    146 2 B2O 6
    150 3 other array 1

    probably I can even do it without a temp table...

    Cheers Rolf
    (addicted to the Perl Programming Language and ☆☆☆☆ :)
    Wikisyntax for the Monastery

      > probably I can even do it without a temp table...

      looks good! :)

      DROP TABLE IF EXISTS t_tree_insert; CREATE TABLE `t_tree_insert` ( `f_node_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `f_parent_id` INT(10) UNSIGNED NULL DEFAULT '0', `f_name` VARCHAR(200) NOT NULL, `f_sort` FLOAT NULL DEFAULT NULL, PRIMARY KEY (`f_node_id`), UNIQUE INDEX `sort` (`f_parent_id`, `f_sort`), INDEX `f_parent_id` (`f_parent_id`), FULLTEXT INDEX `f_name` (`f_name`) ) COMMENT='test tree move operations' COLLATE='utf8_general_ci' ENGINE=InnoDB AUTO_INCREMENT=0 ; insert into t_tree_insert values (130, 1, 'other array', 1) , (142, 2, 'B06', 1) , (143, 2, 'B2L', 2) , (144, 2, 'B2M', 3) , (200, 2, 'INSERT', 3.5) , (145, 2, 'B2N', 4) , (146, 2, 'B2O', 5) , (150, 3, 'other array', 1) ; select * from t_tree_insert order by f_parent_id, f_sort; set @sort=0; replace into t_tree_insert (f_node_id, f_parent_id, f_name, f_sort) select f_node_id, f_parent_id, f_name, (@sort:=@sort+1) as f_sort from t_tree_insert as T where f_parent_id = 2 order by T.f_sort ; select * from t_tree_insert order by f_parent_id, f_sort;

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Wikisyntax for the Monastery

      UPDATE

      But I'm suspecting that temporarily dropping the "unique index" constraint and only updating the row f_sort is considerably faster for larger tables.

Re: [OT] MySQL recalibrating a sort-index
by Anonymous Monk on Feb 17, 2018 at 01:22 UTC
    www.sitepoint.com/hierarchical-data-database/ ... This old post from another site] describes another way to do it – modified pre-order traversal trees (MPTTs). Strangely, I did not immediately identify suport for them in CPAN but they surely must be in there somewhere.
      This is slightly off-topic, since my problem is about rearranging a linear order and not necessarily connected to trees.

      Of course I knew this article already, it deals with structure which can be described as hashes of hashes.

      But my f_sort expands the model to also include arrays.

      Please note that the described downsides are rather obsolete for trees with a maximal depth, as long as you can use Self Joins and Updatable Views.

      And even a dynamic depth could be handled by redefining the view.

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Wikisyntax for the Monastery

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1209251]
Approved by herveus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (4)
As of 2024-04-19 05:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found