Memory allocation errors on 32 bit instances

jonathan.underwood's Avatar


06 Dec, 2017 05:16 PM

I am running a lot of tests for a project (python-lz4). The test harness I am using, pytest, runs all of the tests (numbering 800+) in a single python process. The tests are using a extension module which calls the LZ4 library, and the LZ4 library makes a lot of memory allocations and deallocations (outside of the Python managed heap, but within the process heap i.e. using malloc/calloc/free). What I am seeing is that at a certain point the mallocs begin to fail. It looks like this is because the memory is fragmented. This doesn't affect 64 bit builds. I've looked at possibly breaking the tests up to run in multiple (sequential) processes, but so far haven't found a straightforward way to do that. Has anyone else encountered this problem on Appveyor? Any suggestions for a work around?

  1. 1 Posted by Ilya Finkelshte... on 06 Dec, 2017 06:42 PM

    Ilya Finkelshteyn's Avatar

    Maybe you could look at parallel testing. It is actually different builds (on different VMs), very similar to the build matrix. For free accounts it will run sequentially, but you will get full isolation between groups of tests.

  2. 2 Posted by jonathan.underw... on 06 Dec, 2017 08:20 PM

    jonathan.underwood's Avatar

    Thanks Ilya, I'll have a look at those.

    In the meantime, is there an easy way to specify that the test_script
    section should only be ran for some members of the build matrix?

  3. 3 Posted by Ilya Finkelshte... on 06 Dec, 2017 09:03 PM

    Ilya Finkelshteyn's Avatar

    I think I just answered that here but you can play with environment variables any way you want. For example you can check for specific values of that variable, and do it inside `build.cmd, it is up to you.

  4. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:25 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts


? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac