If you give us more information we should be able to help.
Meanwhile, these nodes might give you some indication of what your problem might be:
Update: some Dev references:
- Profiling Memory Usage talk by Tim Bunce (YAPC::NA Austin Texas 2013 featuring Devel::SizeMe). Resident size vs vm size, rss, segments, /proc/pid/maps (e.g. SO maps question), malloc and the heap, perl internals, pads, constant folding, arenas, recursion, perl data structures (lots of pointers), ...
- Plans for adding memory profiling to Devel::NYTProf