Slow speeds with MozRepl

emav has asked for the wisdom of the Perl Monks concerning the following question:

Here's the situation in my humble monastic cell... I recently bought a Synology NAS, installed Synology Drive and started syncing my drives to have a backup at hand in case an electronic fiend struck down my system.

The thing is that I also use a nice little script I wrote to scan various sites for free e-books and comic strips that I can then download. The e-book script uses WWW::Mechanize::Firefox and MozRepl on Firefox 45.9 while the comics script uses LWP::Simple. The latter runs without a problem at all times. However, whenever the Synology Drive (SD) software is running, my e-book script simply stagnates and can hardly communicate with MozRepl on Firefox. Regularly, they would exchange data at a rate of hundreds of KB/s but, when SD is doing its syncing tasks, the data rate drops to meagre B/s. Even if I pause SD, I see no improvement. The only thing that does the trick is a reboot and making sure SD remains paused while my script is running.

I have tried applying network filters that leave ample bandwidth for Perl and MozRepl to use but to no effect, of course, as they are both running on the same machine. I have tried to limit Synology Drive's CPU usage but that didn't work either. It never appeared to have a voracious appetite for CPU time, anyway. I've read that most syncing apps simply use too much memory to bring our machines to their knees but with 64GB on board I would be surprised to see it go in a flash down SD's drainpipe (problems appear even if SD starts running a couple of minutes before I trigger my script).

So... I am at a loss as to how to go about spotting the problem (let alone solving it). Any pointers anybody?

Comment on Slow speeds with MozRepl

Replies are listed 'Best First'.
Re: Slow speeds with Synology by Anonymous Monk on Jul 15, 2018 at 00:32 UTC
https://www.synology.com/en-global/knowledgebase/DSM/tutorial/General/What_should_I_do_if_file_transfers_are_slow	[reply]
Re^2: Slow speeds with Synology by emav (Pilgrim) on Jul 15, 2018 at 09:23 UTC
Thanks for the reply but that is not the issue. The connection speed between my computer and the NAS is just fine. The connection speed between Firefox and the Internet is not the problem either. The issue here is the connection speed between my perl script and MozRepl on Firefox. That's what Synology Drive seems to affect and reduce to just B/s and I can't figure out how or why this happens.	[reply]
Re^3: Slow speeds with Synology by Anonymous Monk on Jul 15, 2018 at 11:02 UTC
So theyre trying to use the same ports ...?	[reply]
Re: Slow speeds with MozRepl by Marshall (Canon) on Jul 16, 2018 at 05:14 UTC
The e-book script uses WWW::Mechanize::Firefox and MozRepl on Firefox 45.9 The latest versions of Firefox breaks some programs and plugins that I like. I think there is an interface to Chrome that might me better? Don't know. I installed "WaterFox" just so that I could run the SQLite Manager plug-in.	[reply]
Re^2: Slow speeds with MozRepl by Corion (Patriarch) on Jul 16, 2018 at 07:08 UTC
Firefox works with the mozrepl plugin and WWW::Mechanize::Firefox up to version Firefox 56.x. I'm in the process of looking at whether Waterfox is a good replacement but haven't even attempted to run the test suite with it. If it is just a fork of the original Firefox with the occasional security fix backported, then it should work without problems.	[reply]
Re: Slow speeds with MozRepl by Anonymous Monk on Jul 15, 2018 at 08:37 UTC
This does, indeed, sound strange. Various synchronization programs are known to cause significant input/output load, which may slow down other programs if they try to do I/O on the same physical device. Can you measure how much I/O does Synology Drive cause on your system while syncing? Which operating system you are on?	[reply]
Re^2: Slow speeds with MozRepl by emav (Pilgrim) on Jul 15, 2018 at 09:39 UTC
I'm on Win10. Indeed, I/O activity on the hd where Synology Drive is performing sync tasks can reach top levels (even spikes of 100%) but the c: drive where perl, firefox, and my scripts reside has been left out of Synology Drive's reach (I have another program that backs it up when my computer is idle). I guess it might bottleneck at the controller level but would that be so persistent as to cause problems even when the synchronization tasks are paused? There seems to be no I/O activity of this sort at the time. Yet, here we are.	[reply]
Re^3: Slow speeds with MozRepl by aitap (Curate) on Jul 15, 2018 at 10:50 UTC
Unless someone has a better idea, it may make sense to capture execution profiles of runs both with and without Synology Drive active, both Perl and C sides of things. The latter might demonstrate whether Synology Drive is doing something unusual, like injecting its DLL into other processes.	[reply]
Re: Slow speeds with MozRepl by marto (Cardinal) on Jul 15, 2018 at 09:52 UTC
What is the model number of the device? A specification would be interesting.	[reply]
Re: Slow speeds with MozRepl by emav (Pilgrim) on Jul 19, 2018 at 05:54 UTC
Thank you all for you input... More on this issue... It's clearly a Synology Drive thing as, over the past few days, I have observed that the B/s problem appears with my apache server as well, rendering any testing on locally hosted copies of my sites unfeasible. The strange thing though is this: I have a Dreambox satellite receiver that is connected to my network, of course. The program I use to watch SatTV on my computer is DreamStream. Now, when DreamStream is initiated, it downloads a listing of all the channels and bouquets that are available on my Dreambox. We're talking about a couple of XML files not larger than 10KB or so. DreamStream almost hangs there. It's sooo slooow... But, once those files have been downloaded, streaming is spotless. SD channels can stream even when Synology Drive is syncing while HD channels require that I limit Synology Drive's access to my local (wireless) network. So, this is all very strange. I guess that maybe streaming works because of my router configuration, maybe because I'm not such a bad guy after all. In any case, it's clearly a Synology issue, as I've already mentioned. I think I'll drop them a hint and update you with any info they provide. Thanks again!	[reply]


Perl: the Markov chain saw
	PerlMonks