The Cygwin environment used by GAP limits its workspace to 384 MB of memory. The page "GAP Installation on Windows" from the GAP website suggests how to increase the memory limit by adding an appropriate key to the Windows registry. However, the GAP Support group had obtained several reports that the solution recommended there does not work under modern versions of Windows, namely, Windows Vista and Windows7. If you are using Windows Vista or Windows7, you may try to change the registry as described below. So far it worked for me for GAP 4.5 beta on a 32-bit Windows7 with a proper Cygwin installation, and for GAP 4.4.12 on a Windows Vista machine without Cygwin. I'd be very interested to hear whether it works for you as well, including the details of your versions of GAP and Windows, and whether or not you have the Cygwin installed on your machine.

First, you might have to run GAP at least once to create the appropriate registry keys. Then, edit the batch file "usemem.bat" in the "bin" folder, modifying registry keys dependently on your version of GAP:

For GAP 4.4.12, replace "HKLM" by "HKCU" to have

echo "Setting MaxMem registry value"
regtool -i set "/HKCU/Software/Cygnus Solutions/Cygwin/heap_chunk_in_mb" 1024
echo "Registry values for Cygwin are:"
regtool -v list "/HKCU/Software/Cygnus Solutions/Cygwin"

For GAP 4.5 beta, replace "HKLM" by "HKCU" and remove "/Cygnus Solutions" from the path, so you will have

echo "Setting MaxMem registry value"
regtool -i set "/HKCU/Software/Cygwin/heap_chunk_in_mb" 1024
echo "Registry values for Cygwin are:"
regtool -v list "/HKCU/Software/Cygwin"

Now exit all running GAP sessions and other Cygwin processes (if any), run "usemem.bat", and restart GAP (don't forget to specify the value of "-o" option in the bat-file(s) used to start GAP). Check whether your GAP can use more memory now.

Note that the new version of the batch file "usemem.bat" will change the memory limit only for the current user, so if GAP is used in multiple accounts, each user will have to run this batch file separately.

If you wish to change this setting globally, or if the "usemem.bat" trick did not work, you may try to do the change by hand. An example below shows how to do it globally for GAP 4.5 beta. For GAP 4.4.12, use the path with "Cygnus Solutions" as in the first version of this batch file.

1. Start C:/WINDOWS/regedit.exe

2. Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin

3. Select "New" --> "DWORD(32-bit) Value" in the context menu:



4. Enter the name "heap_chunk_in_mb" at the prompt:



5. Select "Modify Binary Data..." in the context menu:



6. Adjust the value as shown and press "OK":



6. Check that you set the correct value:



Now you hopefully will be able to run GAP with larger amount of memory.

Alexander Konovalov,
July 28th, 2011