LanX has asked for the wisdom of the Perl Monks concerning the following question:
Hi
There is an actual Perl context, I'm representing (kind of) a HoAoHoA tree structure in an SQL-table (so it's not totally off-topic).
The array part is using a field f_sort to represent the order.
For instance: (the parent_id=2 identifies a particular array inside the tree)
f_node_id |
f_parent_id |
f_name |
f_sort |
142 |
2 |
B06 |
1 |
143 |
2 |
B2L |
2 |
144 |
2 |
B2M |
3 |
145 |
2 |
B2N |
4 |
146 |
2 |
B2O |
5 |
Now I have the requirement to delete , insert and move ° multiple elements in this "array", which of course involves renumbering f_sort .
I wanted to be clever and avoid thinking about all involved edge cases, and changed the type of f_sort to float, such that I can just move new elements in between two elements as fractions. (the monastery is using a similar concept for sorting in user settings)
f_node_id |
f_parent_id |
f_name |
f_sort |
142 |
2 |
B06 |
1 |
143 |
2 |
B2L |
2 |
394 |
2 |
*Inserted* |
2.5 |
144 |
2 |
B2M |
3 |
145 |
2 |
B2N |
4 |
146 |
2 |
B2O |
5 |
And after each operation I would just run one SQL statement to "refresh" the f_sort index to integers starting from 1.
SET @sort = 0;
UPDATE t_tree AS T
SET f_sort = (@sort := @sort +1)
WHERE T.f_parent_id = 2
ORDER BY f_sort;
Now this fails since (f_parent_id, f_sort) are set to be unique, i.e. numbering the inserted element 394 to f_sort :=3 will clash with the already available element 144 with f_sort=3 (and so on)
Of course I could start here sorting from the end, but this contradicts the intention to avoid creating code for each case.
A workaround is to multiply each f_sort with -1 before re-sorting, because new indices can't clash with negative numbers.
But this feels quite clumsy.
I could achieve the same effect with an extra boolean column f_is_currently_sorting and add it to the "unique index" (it would reflect the "negativity" of -1) ... at least the sort order would stay correct in every phase. But still.
Any better ideas for that approach?
°) move in the sense of cut&paste or copy&paste of multiple elements, be from the same or from another array
update
I was asked
> I'm not clear on what you're wanting to do. Can you add a bit of information on what the sort criteria are?
ORDER BY f_sort WHERE f_parent_id = ?
f_node_id is the primary key
f_parent_id is the node_id of an upper structure
f_name is the key of a hash if f_sort= NULL,
otherwise it's just an optional "name" of a array element
+*
f_sort is the array index and NULL for hashes
so yes the sort criteria is actually
ORDER BY f_sort,f_name WHERE f_parent_id = ?
but the hash logic is not relevant for this question.
*) think sorted hashes, or representing JSON, where some RFC require that the order of keys matters
Re: [OT] MySQL recalibrating a sort-index
by erix (Prior) on Feb 16, 2018 at 11:50 UTC
|
UPDATE: I added an CONSTRAINT DEFERRABLE to the create table, which renders LanX comment obsolete. Sorry about that./UPDATE
This is not for Oracle MySQL, I think (I didn't find the necessary functionality). This is just for fun, therefore Postgres :P
This won't work for you but you expressed some interest (or what I took to be interest) in the CB so here goes:
Below is the (p)sql-inside-bash that I cobbled together; I inserted the database-output in the script ( /* commented out */ ) for better reading.
#!/bin/sh
t=ctetree
echo "
\set insert_location 2.5
begin transaction;
drop table if exists $t ;
create table $t (
f_node_id int primary key
, f_parent_id int
, f_name text
, f_sort float
, constraint tree_parent_sort_uniq_idx unique (f_parent_id, f_sort) d
+eferrable
);
insert into $t values
(142, 2, 'B06', 1)
, (143, 2, 'B2L', 2)
, (144, 2, 'B2M', 3)
, (145, 2, 'B2N', 4)
, (146, 2, 'B2O', 5)
;
\echo -- base data:
table $t;
/* -- base data:
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+--------+--------
142 | 2 | B06 | 1
143 | 2 | B2L | 2
144 | 2 | B2M | 3
145 | 2 | B2N | 4
146 | 2 | B2O | 5
(5 rows)
*/
insert into $t values (394, 2, '*Inserted 1st*', :insert_location);
update $t set f_sort = floor(f_sort)+1 where f_sort >= :insert_locatio
+n;
/* THIS BETTER REPLACED BY... see my later posts downstream */
\echo -- new begin state, after 1st insert and renumbered:
table $t order by f_sort;
/* -- new begin state, after 1st insert and renumbered:
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+----------------+--------
142 | 2 | B06 | 1
143 | 2 | B2L | 2
394 | 2 | *Inserted 1st* | 3
144 | 2 | B2M | 4
145 | 2 | B2N | 5
146 | 2 | B2O | 6
(6 rows)
*/
with inserted as (
insert into $t values (395, 2, '*Inserted 2nd*', :insert_location)
returning f_sort
)
update $t set f_sort = floor(f_sort)+1 where f_sort > (select f_sort
+from inserted);
\echo -- after 2nd insert
table $t order by f_sort;
/* -- after 2nd insert
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+----------------+--------
142 | 2 | B06 | 1
143 | 2 | B2L | 2
395 | 2 | *Inserted 2nd* | 2.5
394 | 2 | *Inserted 1st* | 4
144 | 2 | B2M | 5
145 | 2 | B2N | 6
146 | 2 | B2O | 7
(7 rows)
*/
update $t set f_sort = floor(f_sort)+1 where f_sort <> floor(f_sort);
\echo -- after 2nd insert - renumbered
table $t order by f_sort;
/* -- after 2nd insert - renumbered
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+----------------+--------
142 | 2 | B06 | 1
143 | 2 | B2L | 2
395 | 2 | *Inserted 2nd* | 3
394 | 2 | *Inserted 1st* | 4
144 | 2 | B2M | 5
145 | 2 | B2N | 6
146 | 2 | B2O | 7
(7 rows)
*/
--
-- remove rows 2 and 3, and insert them after the row where f_sort
+is 5.
--
\set location_to_insert 5
\echo -- initial:
table $t order by f_sort;
/* -- initial:
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+----------------+--------
142 | 2 | B06 | 1
143 | 2 | B2L | 2
395 | 2 | *Inserted 2nd* | 3
394 | 2 | *Inserted 1st* | 4
144 | 2 | B2M | 5
145 | 2 | B2N | 6
146 | 2 | B2O | 7
(7 rows)
*/
create temp table _t_$t ( like $t including all );
\echo -- intermediary 1, empty temp table (to receive deleted rows):
table _t_$t order by f_sort;
/* -- intermediary 1, empty temp table (to receive deleted rows):
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+--------+--------
(0 rows)
*/
\echo -- with deleted -> d. rows inserted into tmp table
with deleted as (
delete from $t where f_sort in (2,3)
returning *
)
insert into _t_$t select * from deleted;
\echo -- intermediary 2 (filled with the deleted rows):
table _t_$t order by f_sort;
/* -- intermediary 2 (filled with the deleted rows):
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+----------------+--------
143 | 2 | B2L | 2
395 | 2 | *Inserted 2nd* | 3
(2 rows)
*/
\echo -- with updated -> rows inserted in main table
with updated as (
update _t_$t set f_sort = :location_to_insert + f_sort / 1000
returning *
)
insert into $t table updated;
create temp table _t as
select *, row_number() over () as new_number
from (select * from $t order by f_sort) f
order by new_number
;
\echo -- semi-ready:
table _t order by f_sort;
/* -- semi-ready:
f_node_id | f_parent_id | f_name | f_sort | new_number
-----------+-------------+----------------+--------+------------
142 | 2 | B06 | 1 | 1
394 | 2 | *Inserted 1st* | 4 | 2
144 | 2 | B2M | 5 | 3
143 | 2 | B2L | 5.002 | 4
395 | 2 | *Inserted 2nd* | 5.003 | 5
145 | 2 | B2N | 6 | 6
146 | 2 | B2O | 7 | 7
(7 rows)
*/
update $t t set f_sort = _t.new_number
from _t
where _t.f_node_id = t.f_node_id --> join
and _t.new_number <> t.f_sort --> avoid writes
;
\echo -- ready:
table $t order by f_sort;
/* -- ready:
f_node_id | f_parent_id | f_name | f_sort
-----------+-------------+----------------+--------
142 | 2 | B06 | 1
394 | 2 | *Inserted 1st* | 2
144 | 2 | B2M | 3
143 | 2 | B2L | 4
395 | 2 | *Inserted 2nd* | 5
145 | 2 | B2N | 6
146 | 2 | B2O | 7
(7 rows)
*/
rollback; -- clean up
" | psql -Xqa | less -iSR
UPDATE: slightly tweaked UPDATEs
| [reply] [d/l] |
|
> This is not for Oracle MySQL, I think (I didn't find the necessary functionality).
MariaDB to be precise. (Sorry, I adapted to the sloppy wording of my department.)
> This is just for fun, therefore Postgres :P
Oh, I expected nothing less - than excellence - from you! :P
I'll translate it to a "minor form of SQL", have a deeper look into it, and give feedback.
(Not sure if the floor approach works for multiple changes.)
Thanks! :)
| [reply] |
|
Yeah, the other way is more general: create, then join to a temporary table. (that floor() was just a first try, I should have removed it)
insert into $t values (394, 2, '*Inserted 1st*', :insert_location);
-- update $t set f_sort = floor(f_sort)+1 where f_sort >= :insert_loca
+tion; -- meh
-- instead:
-- create temporary table
create temp table _t as
select f_node_id, f_sort, row_number() over () as new_number
from (
select f_node_id, f_sort
from $t
order by f_sort
) as f
order by new_number ;
-- update table
update $t t set f_sort = _t.new_number
from _t
where _t.f_node_id = t.f_node_id --> join expression
and _t.new_number <> t.f_sort --> avoid unnecessary writes
;
drop table _t; --> remove temp table
It's verbose-ugly but it works without repeating the values.
| [reply] [d/l] |
|
|
|
|
Re: [OT] MySQL recalibrating a sort-index
by choroba (Cardinal) on Feb 16, 2018 at 08:08 UTC
|
What do you mean by "creating code for each case"?
SELECT 1 + COUNT(1)
INTO @sort
FROM t_tree AS t
WHERE t.f_parent_id = 2
LIMIT 1;
UPDATE t_tree AS t
SET f_sort = (@sort := @sort - 1)
WHERE T.f_parent_id = 2
ORDER BY f_sort DESC;
($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
| [reply] [d/l] [select] |
|
| [reply] |
Re: [OT] MySQL recalibrating a sort-index (replace)
by LanX (Saint) on Feb 17, 2018 at 00:23 UTC
|
Hmm .. combining a temporary TABLE with REPLACE seems to do the trick.
DROP TABLE IF EXISTS t_tree_insert, _t;
CREATE TABLE `t_tree_insert` (
`f_node_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`f_parent_id` INT(10) UNSIGNED NULL DEFAULT '0',
`f_name` VARCHAR(200) NOT NULL,
`f_sort` FLOAT NULL DEFAULT NULL,
PRIMARY KEY (`f_node_id`),
UNIQUE INDEX `sort` (`f_parent_id`, `f_sort`),
INDEX `f_parent_id` (`f_parent_id`),
FULLTEXT INDEX `f_name` (`f_name`)
)
COMMENT='test tree move operations'
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=0
;
insert into t_tree_insert values
(130, 1, 'other array', 1)
, (142, 2, 'B06', 1)
, (143, 2, 'B2L', 2)
, (144, 2, 'B2M', 3)
, (200, 2, 'INSERT', 3.5)
, (145, 2, 'B2N', 4)
, (146, 2, 'B2O', 5)
, (150, 3, 'other array', 1)
;
select * from t_tree_insert order by f_parent_id, f_sort;
set @sort=0;
create temporary table _t as
select f_node_id, f_parent_id, f_name, (@sort:=@sort+1) as f_sort
from t_tree_insert as T
where f_parent_id = 2
order by T.f_sort
;
select * from _t;
replace
into t_tree_insert
(f_node_id, f_parent_id, f_name, f_sort)
select * from _t
;
DROP TABLE _t;
select * from t_tree_insert order by f_parent_id, f_sort;
t_tree_insert
130 |
1 |
other array |
1 |
142 |
2 |
B06 |
1 |
143 |
2 |
B2L |
2 |
144 |
2 |
B2M |
3 |
200 |
2 |
INSERT |
3,5 |
145 |
2 |
B2N |
4 |
146 |
2 |
B2O |
5 |
150 |
3 |
other array |
1 |
_t
142 |
2 |
B06 |
1 |
143 |
2 |
B2L |
2 |
144 |
2 |
B2M |
3 |
200 |
2 |
INSERT |
4 |
145 |
2 |
B2N |
5 |
146 |
2 |
B2O |
6 |
t_tree_insert
130 |
1 |
other array |
1 |
142 |
2 |
B06 |
1 |
143 |
2 |
B2L |
2 |
144 |
2 |
B2M |
3 |
200 |
2 |
INSERT |
4 |
145 |
2 |
B2N |
5 |
146 |
2 |
B2O |
6 |
150 |
3 |
other array |
1 |
probably I can even do it without a temp table...
| [reply] [d/l] |
|
> probably I can even do it without a temp table...
looks good! :)
DROP TABLE IF EXISTS t_tree_insert;
CREATE TABLE `t_tree_insert` (
`f_node_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`f_parent_id` INT(10) UNSIGNED NULL DEFAULT '0',
`f_name` VARCHAR(200) NOT NULL,
`f_sort` FLOAT NULL DEFAULT NULL,
PRIMARY KEY (`f_node_id`),
UNIQUE INDEX `sort` (`f_parent_id`, `f_sort`),
INDEX `f_parent_id` (`f_parent_id`),
FULLTEXT INDEX `f_name` (`f_name`)
)
COMMENT='test tree move operations'
COLLATE='utf8_general_ci'
ENGINE=InnoDB
AUTO_INCREMENT=0
;
insert into t_tree_insert values
(130, 1, 'other array', 1)
, (142, 2, 'B06', 1)
, (143, 2, 'B2L', 2)
, (144, 2, 'B2M', 3)
, (200, 2, 'INSERT', 3.5)
, (145, 2, 'B2N', 4)
, (146, 2, 'B2O', 5)
, (150, 3, 'other array', 1)
;
select * from t_tree_insert order by f_parent_id, f_sort;
set @sort=0;
replace
into t_tree_insert
(f_node_id, f_parent_id, f_name, f_sort)
select f_node_id, f_parent_id, f_name, (@sort:=@sort+1) as f_sort
from t_tree_insert as T
where f_parent_id = 2
order by T.f_sort
;
select * from t_tree_insert order by f_parent_id, f_sort;
UPDATE
But I'm suspecting that temporarily dropping the "unique index" constraint and only updating the row f_sort is considerably faster for larger tables. | [reply] [d/l] |
Re: [OT] MySQL recalibrating a sort-index
by Anonymous Monk on Feb 17, 2018 at 01:22 UTC
|
www.sitepoint.com/hierarchical-data-database/ ... This old post from another site] describes another way to do it – modified pre-order traversal trees (MPTTs). Strangely, I did not immediately identify suport for them in CPAN but they surely must be in there somewhere. | [reply] |
|
This is slightly off-topic, since my problem is about rearranging a linear order and not necessarily connected to trees.
Of course I knew this article already, it deals with structure which can be described as hashes of hashes.
But my f_sort expands the model to also include arrays.
Please note that the described downsides are rather obsolete for trees with a maximal depth, as long as you can use Self Joins and Updatable Views.
And even a dynamic depth could be handled by redefining the view.
| [reply] |
|
|