Do you have some example URLs? How many do you have? Can you show an example of the processing routine? Perhaps Mojo::UserAgent in conjunction with Mojo::Promise (see the Mojo::UA example) or Mojo::IOLoop would help runtime.