New method to detect total available RAM on the building machine by valtandor · Pull Request #1623 · easybuilders/easybuild-framework

valtandor · 2016-02-18T20:38:57Z

Required by easybuilders/easybuild-easyblocks#834

hpcugentbot · 2016-02-18T20:38:58Z

Automatic reply from Jenkins: Can I test this?

boegel · 2016-02-18T20:52:21Z

easybuild/tools/systemtools.py

start with None?

boegel · 2016-02-18T20:52:31Z

easybuild/tools/systemtools.py

if memtotal is None?

valtandor · 2016-02-18T20:54:37Z

@boegel: Done.

boegel · 2016-02-18T20:54:50Z

@valtandor: are you up for writing a simply unit test for this (in test/framework/systemtools.py)?

Just something that checks whether a non-zero memory size is returned on Linux, and raises the exception on Darwin?

If not, that's fine, I can help with that.

Where are you planning to use this?

Maybe also add it to get_system_info?

valtandor · 2016-02-18T20:55:58Z

@boegel: This was used in the NWChem EasyBlock that I wrote. Part of the compilation involves trying to get the amount of available system memory. If get_system_info is a better place for it, I'm OK with that.

Where do the unit tests go?

boegel · 2016-02-18T21:10:15Z

In get_system_info, you would just add a call to the function you wrote here, the function itself looks perfect.

Unit tests for functions in systemtools.py go in https://github.com/hpcugent/easybuild-framework/blob/master/test/framework/systemtools.py .

Take a look at the test_glibc_version* tests, you could write a group of tests for your version in a very similar way.

boegel · 2016-02-18T21:18:35Z

Jenkins: ok to test

hpcugentbot · 2016-02-18T22:24:55Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2730/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-22T00:29:42Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2746/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-24T02:32:43Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2765/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

…termined

boegel · 2016-02-24T19:12:07Z

@valtandor: please take a look at valtandor#2, which updates this PR a bit (implement support for Darwin + enhances tests)

fgeorgatos · 2016-02-24T19:24:34Z

easybuild/tools/systemtools.py

+    """
+    Try to ascertain this node's total memory
+
+    @return: total memory as an integer, specifically a number of megabytes


If possible, it would be nice, if there was the ability to tune things from the outside about this, such as $BC_MEM_PER_NODE:
http://centers.hpc.mil/consolidated/bc/policies.php?choice=environment

The meaning of this is to be able to modify builds behavior dynamically, by adjusting such a variable.

If we do that, we should do it consistently (also for other system features), and it should be well documented which environment variables EB picks up on...

So, not in this PR.

boegel · 2016-03-09T15:38:07Z

@valtandor ping on merging valtandor#2 in here?

split up tests for get_total_memory, fix implementation on Darwin

hpcugentbot · 2016-03-15T02:42:19Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2868/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-03-15T03:06:53Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2869/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

boegel · 2016-03-30T09:05:25Z

Jenkins: test this please

hpcugentbot · 2016-03-30T09:15:40Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2922/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

boegel · 2016-03-30T09:26:13Z

@valtandor broken test is fixed by valtandor#3, let's get this merged...

fix mocking of 'sysctl -n hw.memsize' in systemtools tests

hpcugentbot · 2016-03-30T23:16:16Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2924/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-31T15:20:27Z

Going in, thanks @valtandor!

Benjamin Roberts added 14 commits November 10, 2015 15:32

Added method to detect system memory

02bf52c

Fixed some bugs

4aaf51f

Merge branch 'runcmd_fix' into availmem

f9d6c2b

Merge branch 'develop' into availmem

71bce32

Merge branch 'develop' into availmem

15fd6cf

Merge branch 'develop' into availmem

6856184

Merge branch 'develop' into availmem

68a56c0

Merge branch 'develop' into availmem

a6ef955

Merge branch 'develop' into availmem

1535244

Merge branch 'develop' into availmem

b5cd01e

Merge branch 'develop' into availmem

0f3ce85

Merge branch 'develop' into availmem

5336fbf

Merge branch 'develop' into availmem

c509cb3

Merge branch 'develop' into availmem

1d84a68

boegel added this to the v2.7.0 milestone Feb 18, 2016

boegel reviewed Feb 18, 2016
View reviewed changes

easybuild/tools/systemtools.py Outdated

Copy link

Member

boegel Feb 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start with None?

Merge branch 'develop' into availmem

e4a45d0

boegel reviewed Feb 18, 2016
View reviewed changes

easybuild/tools/systemtools.py Outdated

Copy link

Member

boegel Feb 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if memtotal is None?

Replace initial value of 0 with None

d7c7b6f

boegel mentioned this pull request Feb 18, 2016

Updates to NWChem easyblock for version 6.6 etc. easybuilders/easybuild-easyblocks#834

Open

valtandor mentioned this pull request Feb 18, 2016

Files for building recent NWChem releases with EasyBuild, and patches for same easybuilders/easybuild-easyconfigs#2512

Closed

Merge branch 'develop' into availmem

d18aedb

boegel added 3 commits February 24, 2016 18:50

don't raise SystemException, return UNKNOWN if memory could not be de…

f1f6909

…termined

break up test for get_total_memory in tests for Linux, Darwin and native

9efefd3

fix implementation of get_total_memory on Darwin

5ebe8da

boegel mentioned this pull request Feb 24, 2016

split up tests for get_total_memory, fix implementation on Darwin valtandor/easybuild-framework#2

Merged

fgeorgatos reviewed Feb 24, 2016
View reviewed changes

Benjamin Roberts added 2 commits March 1, 2016 16:11

Merge branch 'develop' into availmem

ce0fb44

Merge branch 'develop' into availmem

dd14902

boegel modified the milestones: v2.7.0, v2.8.0 Mar 11, 2016

Benjamin Roberts and others added 2 commits March 14, 2016 12:13

Merge branch 'develop' into availmem

2e68482

Merge pull request #2 from boegel/availmem

0c709ac

split up tests for get_total_memory, fix implementation on Darwin

Benjamin Roberts added 2 commits March 15, 2016 15:53

Merge branch 'develop' into availmem

60bb532

Merge conflicts from Kenneth's PR

042b1ae

fix mocking of 'sysctl -n hw.memsize' in systemtools tests

f4f34df

boegel mentioned this pull request Mar 30, 2016

fix mocking of 'sysctl -n hw.memsize' in systemtools tests valtandor/easybuild-framework#3

Merged

Merge pull request #3 from boegel/availmem

3a72d29

fix mocking of 'sysctl -n hw.memsize' in systemtools tests

boegel merged commit 4254466 into easybuilders:develop Mar 31, 2016

Conversation

valtandor commented Feb 18, 2016

Uh oh!

hpcugentbot commented Feb 18, 2016

Uh oh!

boegel Feb 18, 2016

Choose a reason for hiding this comment

Uh oh!

boegel Feb 18, 2016

Choose a reason for hiding this comment

Uh oh!

valtandor commented Feb 18, 2016

Uh oh!

boegel commented Feb 18, 2016

Uh oh!

valtandor commented Feb 18, 2016

Uh oh!

boegel commented Feb 18, 2016

Uh oh!

boegel commented Feb 18, 2016

Uh oh!

hpcugentbot commented Feb 18, 2016

Uh oh!

hpcugentbot commented Feb 22, 2016

Uh oh!

hpcugentbot commented Feb 24, 2016

Uh oh!

boegel commented Feb 24, 2016

Uh oh!

fgeorgatos Feb 24, 2016

Choose a reason for hiding this comment

Uh oh!

boegel Feb 27, 2016

Choose a reason for hiding this comment

Uh oh!

boegel commented Mar 9, 2016

Uh oh!

hpcugentbot commented Mar 15, 2016

Uh oh!

hpcugentbot commented Mar 15, 2016

Uh oh!

boegel commented Mar 30, 2016

Uh oh!

hpcugentbot commented Mar 30, 2016

Uh oh!

boegel commented Mar 30, 2016

Uh oh!

hpcugentbot commented Mar 30, 2016

Uh oh!

boegel commented Mar 31, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Comments