Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

2 Questions on Images and Mod_perl mem usage

by Anonymous Monk
on Jul 01, 2005 at 22:55 UTC ( [id://471833]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello!

I am trying to run multiple websites that all use the same perl script I made. It runs under mod_perl but I am starting to have some bandwidth usage and mem usage problems as my site is holding 150+ people fairly constantly during the day.

Pages That are displayed show many pictures that individual users upload, these pictures then can be clicked on and it shows the image in full size. Is there a way to make perl reduce the image size by making a second set of pictures at thumbnail size? I think this would significantly reduce my bandwidth usage.

My other problem is my script uses 8.5mb per client in mod_perl apache. I use a lot of the same variables in different subs. Would it be more memory efficent to just make these variables globals, instead of declaring them in several different subs, even though they are only used in the sub they are declared at?

Any help is appreciated.

Thanks Much!
  • Comment on 2 Questions on Images and Mod_perl mem usage

Replies are listed 'Best First'.
Re: 2 Questions on Images and Mod_perl mem usage
by perrin (Chancellor) on Jul 02, 2005 at 01:53 UTC
    You should run a proxy server in front of your mod_perl server so that the large processes will not be tied up serving images. Take a look at the information here for help on how to do this.
      perrin++. This is huge. You might be better off with 1500 apache processes and 150 apache + perl processes, rather than 700 apache + perl (numbers depend on situation). The danger with any mass-market application is that you're going to have a bunch of your bulky processes waiting around to push those 2 MB images down a 56k line.
Re: 2 Questions on Images and Mod_perl mem usage
by fmerges (Chaplain) on Jul 02, 2005 at 00:24 UTC

    Hi,

    Erm, you should use thumbnails, for every photo uploaded you make a small thumbnail on the way. I recommend you not doing resizing on the fly when a person is viewing the page, because it's slow once you have a lot of pages/users. On the other side if you have a photo of 2mb, and you show this image on a <img> defining smaller width and height, the server send the 2mb size image to the client, and the browser will do the "scaling"... Take a look at projects around treating the same situation.

    Imager and Image::Magick are great modules for doing all this kind of stuff (changing images and so).

    I have seen other approaches instead of creating and managing the thumbnails, was using Mason caching the thumbnails, so the first time it takes a few ms. more but once you have a cached thumbnail, it's like having the file normally around on your filesystem. See here.

    I can't tell you if this is normal, since I don't see your code, but normally you can do some optimisations... ;-)

    I think that you should write clean code, since memory is cheap, and today there is no problem to have 512mb or 1024mb on your webserver. Using mod_perl: than more memory than better.

    Regards,

    |fire| at irc
      Thanks, I am looking at the image magick stuff, lil confused but ill play with it :)

      My webserver has 6gigs of memory, and is setup for 700 max clients. With all the sites combined I have a traffic load of about 1000 people at any given time. It still runs fast even though it is queing people but looking at my server stats, I have a nice steady increase of traffic every week for past year, so being able to have more max clients would defiantly help.

      An Example

      print &first;
      print &second;

      sub first() {
      my $key = "1";
      return $key;
      }

      sub second() {
      my $key = "2";
      return $key;
      }

      If I made $key a global would it be more efficent memory wise?

        Before taking time reading Image::Magick take a look at Imager first... ;-)

        Wow, our corporate webserver has 512 Mb running mod_perl + Mason.

        The example you put, you don't need to create this vars... but anyway, I understand what you mean, but you must see your code, and think about what optimisation you can do. I like to use global variables only where is necessary or is a really benefit.

        There are modules where you can see where is consuming more time, etc., take a look for Devel::.

      Mogrify, which comes with ImageMagick, resizes images.
Re: 2 Questions on Images and Mod_perl mem usage
by Xaositect (Friar) on Jul 01, 2005 at 23:50 UTC

    Regarding your first question, Image::Magick will handle resizing the images. It's a real pain to get working on Windows I've heard, but it's pretty industrial-strength. You might also try Image::Resize - I haven't used it, but the pod makes what you need look pretty easy to do.


    Xaositect - Whitepages.com
Re: 2 Questions on Images and Mod_perl mem usage
by CountZero (Bishop) on Jul 02, 2005 at 09:21 UTC
    You should really read up on mod_perl and how to optimize it. Practical mod_perl is IMHO the best way to start. It gives a lot of real practical info and speaks extensively on memory usage, speed trade-offs, ...

    I'm just guessing here, but are you running your scripts as ModPerl::Registry scripts? There might be a speed and memory bonus in changing your scripts into real Apache-mod_perl modules.

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

      If we are talking about one script file that is over 10,000 lines long, it needs to be split up just to make it maintainable.

      You could use plain mod_perl modules, an application server framework like Mason might be a good way to simplify your code. These tools can handle a lot of basic application functionality for you.

      It sounds like this project has grown to the point where a major rewrite is necessary. Good luck.


      TGI says moo

Re: 2 Questions on Images and Mod_perl mem usage
by Anonymous Monk on Jul 02, 2005 at 01:36 UTC
    I started this project lil over 2 years ago. At the time I was just learning perl and started my script as kinda a learning experience. I later got some offers to build a website around it and it grew into something very profitable for me. It has grown to a point where I have to start paying for additional bandwidth(why I want thumbnails made auto now) and additional memory for more clients. I have worked on it quite a bit in the past couple years as I keep getting more knowledgable in how to use it. If 8.5mb is the best I can do, I will have to deal with that but if I can get it down even a lil it multiplies so many times over and I wont have to worry about max client availability as much.

    My Script is just one big long script instead of in packages which is a lil part of my newbishness. The only thing seperate from the main script is the script to upload pictures from the websites. I will defiantly look into the imagemagick and imager and I am hopeful that will significantly reduce my bandwidth cost. I guess for the amt of mem used I may need to write some test perl scripts to see how they react so I can tune mine.

    I am a newbie and do not know what you mean by DB. If it is related to data files, I use my own subs to handle my data files with the use of crypt function.

    I have looked at catalyst webpage just now and am not sure if that is really what im looking for but I will read up on it more.

    Thank you very much for the help and any additional advice you have is most welcome. :)

      Hi again,

      Take a look at this book (Practical mod_perl) is really good, teach you how mod_perl work, how code well mod_perl programs, and also method to gain speed when you have a very heavy webapp., using proxy methods, etc...

      If I were in your position, I would think about doing a real refactoring to your webapp. read that book, and making the application run has a handler, making your own packages etc. Or use thinks like Mason or TT or doing some more research and get your fingers into Catalyst and the MVC model. This is the way things are going...

      On the way, getting a introduction in XMLHttpRequest or commonly known as AJAX, to make the interaction with your clients more intuitive and cooler of course.

      With DB I mean DataBase, RDB... using DB to store the users info, accounts, settings, images (path, comment about picture, EXIM data, etc...).

      This way, you can handle more clients, and maintain the ones you already have. Updating your offer, performance boost, more user interaction, and could be a new look'n'feel.

      Use all that you learnt in this 2 years, and learn new things also as a learning experience ;-)

      Regards,

      fmerges

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://471833]
Approved by Ido
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2024-04-23 06:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found