Ah, you were looking for a DEFERRABLE CONSTRAINT, which postpones the constraint-validation to the end of a transaction.
So, you'd do (again, postgres):
create table tree (
f_node_id int primary key
, f_parent_id int
, f_name text
, f_sort float
, constraint tree_parent_sort_uniq_idx unique (f_parent_id, f_sort) d
This lets you mess about with non-unique states for the duration of a transaction. The constraint (here: uniqueness) is only then enforced.
I can't imagine MariaDB does not have this functionality but I can't find it in the documentation. (I looked for DEFERRED or DEFERRABLE. The MariaDB docs list 'DEFERRED' as a reserved word but I see no functionality associated with it.)