Sigasi keeps compiling everything while I am trying to configure my libraries. Things get really slow.

While you are configuring your libraries (mapping and unmapping files), the built-in compiler may trigger a full rebuild several times.

In order to avoid this, and make Sigasi more responsive while you configure your libraries, you can turn of automatic building:
Uncheck Project > Build Automatically. Don't forget to turn it back on after you're done configuring your libraries.