High memory usage
Dear Sigasi Team
I found that sigasi is using a lot of RAM memory which I guess would be much better with a correct memory handling.
Let me try to explain this in the following example:
When a project is open and beeing rebuilt, the RAM usage of HDT is increasing even when no changes on the code are done and beeing rebuilt again.
Also when closing a project in the sigasi project explorer, the RAM usage stays at the same level.
Wouldn't it be possible to clear the old RAM contents when rebuild a project and also when a project is closed? In the moment I work around this problem by closing and opening again sigasi from time to time.
Thanks for your help
- Forums:
Hi Simon, we've looked at
Hi Simon, we've looked at this problem after another report a few weeks back. We looked at it again today and we cannot reproduce the memory leak.
I suppose from you explanation that a full rebuild does not fee up your memory?
Is there a certain operation that causes the memory increase? Do you use external libraries? Any chance we can take a look at your project? Can you Can you send us your error log?
Huge memory needed for Rebuild
Hi,
I actually reported a similar problem a while ago, but I want to report a different memory problem. I normally use sigasi on my desktop computer which has 8GB memory. I'm trying to use it on my laptop which has 4GB memory and the project build gets only 50% of the way through before running out of memory. The OS will only allow me to dedicate 1400 MB of memory to eclipse. The project I'm trying to build is the LEON3 project - so it is large - but it seems to require a huge amount of memory (more than I have to give it). Is there some way to get around this issue?
Nathan.
Hi Nathan, I have noticed
Hi Nathan,
I have noticed that the tech libraries in GRLIB are huge. Nobody in their right mind would want to compile all of them. You usually need just one.
What I did was ignore the entire techmap and tech directories and then include a few specific files that are needed in the rest of the project:
Does that work for you?
Nope it doesn't work
Philippe,
Today I decided to update my Sigasi software to the latest version (Sigasi HDT Feature 1.0.0.20110214114226-78--7RDKKDRKeKKeRRs - updated from version: Sigasi HDT Feature 1.0.0.20110131145027-78--7RDKKDKYKKelDRz). Even with 5GB memory dedicated to Eclipse, I couldn't compile my project without an out of memory error. Even when I gave it 6GB of memory it took over 5 minutes to build. I know that a recent post indicated that memory issues have been found and will be fixed in an upcoming release, but I wanted to clarify what you meant in your last suggestion. You mentioned that you "ignore" all techmap directories but 3. In my "Libraries" configuration, I "exclude" all directories that I don't use (not just in the techmap library) - I actually even exclude the Xilinx unisim directory even though I do use it (so it doesn't recompile every few seconds). However, in order to try to get my project to compile I "removed" all the libraries I don't use.
I guess what I'm asking is - what can I do to further reduce the amount of memory required? Will "removing" libraries I don't even use help? "Excluding" them didn't seem to affect the amount of memory needed to build on startup, although it does seem to affect what gets rebuilt on the fly. Do I need to just be patient for the next release? (if so I'll just roll back to a previous version in the meantime)
Thanks!
Nathan.
Default work library
As I've looked into this more, I noticed a library (among about 40 that are in this project) that I'm pretty sure I didn't create, and that can't be deleted. It's a work directory that points to the base of the project. I noticed that libraries were being built that I explicitly marked for exclusion - but if there's a work library that is all inclusive it will try to build everything. I'm sure I didn't create this library and I'm not able to remove it. Is this automatically created? Do you know why I can't remove it?
Nathan.
Hi Nathan, can you email us
Hi Nathan, can you email us the .libraries.xml file and the CSV file that Sigasi generates? That will give us an idea of how your libraries are configured.
I've been playing with GRLIB last week, so I would not expect any trouble if everything is set up correctly.
Improvements in the upcoming release
Hi,
We have investigated the memory usage and we found some issues. The memory leak should be fixed in the next release.
Lieven
Memory usage spikes when "cleaning" with multiple projects open.
I've noticed the same thing. Whenever I rebuild a project, or "clean" a project (with multiple other referenced projects open), the memory usage rises in every case, and does not release it until I close and restart Sigasi. However, when I close all other projects, and rebuild or clean the single open project, the memory utilization stays the same. The difference seems to be in having one project open, versus many open simultaneously.
The only external library I am using right now is the Xilinx unisim library. If I have multiple (very small) projects open, the memory demands of Sigasi are well over 1GB. This simply should not be.
Hope this helps.
new release / how to measure memory usage
Hi Brad,
We have just released a new version with several memory usage improvements. Our colleague, Lieven, has written a blog post on how to analyse memory issues.
Which method did you use for measuring the memory usage? Can you let us know if you see any improvement with the new release?
kind regards
Philippe
Memory on the new release
Philippe,
I updated SIGASI on Eclipse (64-bit) and followed the suggestions on the blog. According to the Eclipse heap window, it still required a heap size of 5024M and took about 10 minutes to rebuild. After the build finished, the heap indicates that it's using 3021M out of 5014M. I'm pretty sure the reason it takes so much time and space is because of the default "work" library that I can't remove. The library points to "project:/" which is obviously a bad thing since it builds absolutely everything in the project. I tried pointing it to some other project directory, but it got reset to "project:/". I use both the Eclipse and standalone version. I haven't updated the standalone version in a long time (since it "works" and I don't want to touch it till I know the Eclipse version works), and the standalone version doesn't have the default "work" library problem (and takes a couple of minutes and less than 2GB memory to build).
Exclude the folders that you don't use
hi Nathan,
I believe that you are compiling a lot of files that you don't need to (or want to) compile.
We changed the way Sigasi handles libraries a few releases ago. It used to be possible that parts of your project were 'unmapped', meaning they were neither included in a library nor explicitly excluded. This caused unexpected behaviour for a lot of users (eg. you add a file to your project and it isn't compiled while it doesn't show the excluded icon marker). In our current version, everything is included by default in the 'library' work, if you want to exclude source files you have to do it explicitly (see http://www.sigasi.com/documentation/user/libraries.html). Make sure that all the folders that you don't need have the stop sign marker.
Locally I have working version of Grlib, it consumes about 500 MB of memory, including all of the tech libraries.
I hope this helps,
Lieven
Reply to Philippe on 3-14-2011
Philippe,
I checked out the link from Lieven on how to analyze memory issues. After updating Sigasi, I seem to be able to replicate the problem, although it is not as pronounced as it was before the update. Here's what I am seeing.
When triggering a full rebuild, the heap utilization jumps dramatically (presumably due to the massive amount of data structures that Sigasi is rebuilding, or more likely duplicating.) When the utilization gets close to the max heap size, the jvm sends a request to windows (presumably malloc?) to be able to increase the heap size. Enough garbage collection may not be performed before the next build (either manually or automatically) so this heap size may need to increase further to compensate to hold successive builds. The problem with this is that Sigasi (being a wrapped version of eclipse) does not inherently release the unused heap memory back to windows in a reasonable amount of time, when the heap utilization decreases after a transient full rebuild following an eventual garbage collection. In my case, I end up with an ever-increasing memory utilization reported in Windows Task manager, even though most of that memory may unused heap-space by the time the garbage collection rolls around. In some cases, my heap size has dynamically increased to 1Gb as a result of transient rebuilds, even though it may average using 100Mb after garbage collection.
I have found a link that shows how to dynamically release unused heap-space of eclipse back to windows by configuring the eclipse.ini. I assume the process is similar with the stand-alone version of Sigasi, so I will try it and get back to you. Perhaps you might have some suggestions on releasing the memory back to Windows. In the meantime, here is that link:
http://stackoverflow.com/questions/3776041/eclipse-release-heap-back-to-...
I hope this feedback is helpful, because Sigasi is amazing. I absolutely love working with it, and only want to see it improve.
-Brad
Reduce the maximum heap size
In your case the solution is pretty simple, check how much memory you need when all your projects are open after you performed a manual garbage collection. Take that size, double it and set the -Xmx option to that value.
For efficient garbage collection you need the extra memory space space. But in your case I'm guessing you only need about 200-250 MB, in that case allocating 1 GB is a bit excessive.
In the stackoverflow article they talk about using the G1 garbage collector in the accepted answer. I can't recommend that until it is stable.
I hope this helps
Lieven
Out of memory during unisim_VCOMP compilation
Hello,
I've got a similar problem. In my are 2 – 3 sigasi 1.0 projects open and a few Java and CDT ones. My vhdl projects use the Xilinx unisim library, so it is an external library.
From time to time (about 2 times per hour) my eclipse gui stops reacting and after a couple of minutes I got this message: "Out of memory during file compilation: file:/opt/Xilinx/13.1/ISE_DS/ISE/vhdl/src/unisims/unisim_VCOMP.vhd
GC overhead limit exceeded"
This is really annoying! The file unisim_VCOMP.vhd has a size of 1.1 mb, so I can imagine it takes a bit more space in memory. Can I somehow stop sigasi to parse this file (several times)?
My eclipse.ini on my linux 64 system looks like this:
-startup
plugins/org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.2.R36x_v20101019_1345
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-product
org.eclipse.epp.package.linuxtools.product
--launcher.defaultAction
openFile
-vmargs
-Dosgi.requiredJavaVersion=1.6
-Xms40m
-Xmx384m
-XX:MaxPermSize=256m
Any help would be great! It is hardly possible to work with this behavior.
Fabian
bigger heap size
Hi Fabian,
I agree that this is not good behavior.
The compiled internal representation is (a lot) bigger than the file size. Some operations require a complete rebuild of the project. (Still no excuse to block your computer though.)
You should set the heap space to at least 1024m.
As a side-note: Sigasi 2.0 will have better memory-behavior after the initial compilation.
Post new comment