diff options
| author | Simon Derr <Simon.Derr@bull.net> | 2005-08-05 06:52:03 +0400 |
|---|---|---|
| committer | Linus Torvalds <torvalds@g5.osdl.org> | 2005-08-05 08:43:14 +0400 |
| commit | 2f60f8d3573ff90fe5d75a6d11fd2add1248e7d6 (patch) | |
| tree | 173980b12459aa55586d3f4f56630abf5af1ffcc /mm | |
| parent | b68e9f857271189bd7a59b74c99890de9195b0e1 (diff) | |
| download | linux-2f60f8d3573ff90fe5d75a6d11fd2add1248e7d6.tar.xz | |
[PATCH] __vm_enough_memory() signedness fix
We have found what seems to be a small bug in __vm_enough_memory() when
sysctl_overcommit_memory is set to OVERCOMMIT_NEVER.
When this bug occurs the systems fails to boot, with /sbin/init whining
about fork() returning ENOMEM.
We hunted down the problem to this:
The deferred update mecanism used in vm_acct_memory(), on a SMP system,
allows the vm_committed_space counter to have a negative value.
This should not be a problem since this counter is known to be inaccurate.
But in __vm_enough_memory() this counter is compared to the `allowed'
variable, which is an unsigned long. This comparison is broken since it
will consider the negative values of vm_committed_space to be huge positive
values, resulting in a memory allocation failure.
Signed-off-by: <Jean-Marc.Saffroy@ext.bull.net>
Signed-off-by: <Simon.Derr@bull.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Diffstat (limited to 'mm')
| -rw-r--r-- | mm/mmap.c | 6 | ||||
| -rw-r--r-- | mm/nommu.c | 6 |
2 files changed, 10 insertions, 2 deletions
diff --git a/mm/mmap.c b/mm/mmap.c index da3fa90a0aae..404319477e71 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -143,7 +143,11 @@ int __vm_enough_memory(long pages, int cap_sys_admin) leave 3% of the size of this process for other processes */ allowed -= current->mm->total_vm / 32; - if (atomic_read(&vm_committed_space) < allowed) + /* + * cast `allowed' as a signed long because vm_committed_space + * sometimes has a negative value + */ + if (atomic_read(&vm_committed_space) < (long)allowed) return 0; vm_unacct_memory(pages); diff --git a/mm/nommu.c b/mm/nommu.c index ce74452c02d9..fd4e8df0f02d 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -1167,7 +1167,11 @@ int __vm_enough_memory(long pages, int cap_sys_admin) leave 3% of the size of this process for other processes */ allowed -= current->mm->total_vm / 32; - if (atomic_read(&vm_committed_space) < allowed) + /* + * cast `allowed' as a signed long because vm_committed_space + * sometimes has a negative value + */ + if (atomic_read(&vm_committed_space) < (long)allowed) return 0; vm_unacct_memory(pages); |