This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
http://www.kuro5hin.org/?op=comments&sid=2001/1/9/73016/34728&pid=51#55
Memory reporting (3.63 / 11) (#15)
by mcelrath (mcelrath+kuro5hin@draal.physics.wisc.edu) on Tue Jan 9th,
2001 at 11:19:32 AM EST
(User Info) http://draal.physics.wisc.edu/FilterProxy/
I think this perception that X is bloated is a symptom of the fact
that memory reporting under linux just plain sucks. Consider:
1. mmap()'ed memory is added to the application's SIZE
2. shared libraries are added to the application's SIZE, and there's
no straight forward way to determine if more than one application
is using a given shared library (via top/ps, though you could
'lsof | grep')
3. If an application forks, both the parent and child report the same
SIZE initially, despite the fact that linux uses copy-on-write,
and most of that memory is shared between them.
4. threaded applications receive one PID per thread, and each PID
reports the same SIZE, even though all PID's are sharing that
memory.
Taken together, these things tend to lead to very skewed perceptions
of the amount of memory an application is using. Everyone go run 'top'
and type f within it. Notice the number of memory related fields you
can turn on. (%MEM, TSIZE, DSIZE, SIZE, TRS, SWAP, SHARE, RSS, LIB --
there are probably others). But even none of these will tell you how
much memory is mmap()'ed. 'top' and 'ps' both report the amount of
address space in use, not the amount of memory being used. The latter
number is somewhat hard to come by. I, personally, would like to see
this shortcoming addressed. There should be some memory field via
top/ps that when you take all the processes and add this field up, you
get thea amount of used memory. (Mem used + Swap used at the top of
'top') One of my machines amusingly reports that X is using 302MB, on
a 256MB machine, with no swap used!
--Bob
===
RSS+SWAP-(SHARE+LIB) ? (4.25 / 4) (#21)
by molo on Tue Jan 9th, 2001 at 12:04:38 PM EST
(User Info)
You make some good points. However, if you add RSS (resident set size)
and SWAP, then subtract LIB amd SHARE, you should get the amout of
unique memory a task is taking up.
From the top man page:
SWAP Size of the swapped out part of the task.
LIB Size of use library pages. This does not work for ELF
processes.
RSS The total amount of physical memory used by the task,
in kilobytes, is shown here. For ELF processes used
library pages are counted here, for a.out processes
not.
SHARE
The amount of shared memory used by the task is shown
in this column.
I have the following numbers for X (4.0.1, 1600x1200x16, nv module):
SIZE: 83372
RSS: 23M
SWAP: 9.9M
SHARE: 2304
LIB: 0
(23 * 1024) + (9.9 * 1024) - 2304 = 31385
This yields a much more reasonable 30.6M rather than the 81.4M given
by the SIZE field. Perhaps we should call this the 'real' size and add
it to top?
This doesn't help us with the thread issue though. As long as threads
are treated as processes (due the the functioning of clone() ) I don't
know if there will be a simple solution.
--
Whenever you walk by a computer and see someone using pico, be kind.
Pause for a second and remind yourself that: "There, but for the grace
of God, go I." -- Harley Hahn
===
But what about video memory? (4.00 / 3) (#28)
by fluffy grue (#FF00FF @ trikuare . cx) on Tue Jan 9th, 2001 at
12:49:33 PM EST
(User Info) http://trikuare.cx
That figure still probably includes memory-mapped but
non-system-memory things like the AGP window, which depends on how
large it's set in your BIOS (likely anywhere from 32M to 128M), and
the framebuffer, which is (IIRC) done by effectively mmap()ing the PCI
device. I know that when i start up an OpenGL application, the SIZE of
my X server grows from 56M to 192M (128M AGP window), but neither
SWAP, SHARE nor LIB grow at all.
---
"'Is not a quine' is not a quine" is a quine.
Quine "quine?" "'Quined' quined" quines "quined."
===
What about RSS? (3.00 / 3) (#31)
by molo on Tue Jan 9th, 2001 at 01:02:02 PM EST
(User Info)
RSS is supposed to be the "physical memory used." Does this include
mmap()'ed blocks? I'm really not sure. As for SIZE growing by 100+M,
does RSS do so as well? That'd be more interesting to me.
Just so I'm sure you and I are on the same page here, note that SIZE
is not in my little equation at all.
--
Whenever you walk by a computer and see someone using pico, be kind.
Pause for a second and remind yourself that: "There, but for the grace
of God, go I." -- Harley Hahn
[ Parent ]
===
They both grow (3.00 / 2) (#44)
by fluffy grue (#FF00FF @ trikuare . cx) on Tue Jan 9th, 2001 at
04:31:33 PM EST
(User Info) http://trikuare.cx
Both RSS and SIZE grow. Interestingly enough, this time when I ran a
GL program to test this, neither of them grew by 128M (only 10M this
time)... weird.
Oh, wait, it was xlock which was getting the 160M+ size stats, not X.
That's right, it's the DRI's libGL which locks AGP now, not the X
server... must have overlooked that when I was testing last time. The
X server still has to lock more video memory for the framebuffer and
Zbuffer, though, which would explain that increase in size. X is
growing about 10M for both SIZE and RSS.
---
"'Is not a quine' is not a quine" is a quine.
Quine "quine?" "'Quined' quined" quines "quined."
===
Ah! (3.00 / 1) (#51)
by molo on Tue Jan 9th, 2001 at 05:51:33 PM EST
(User Info)
Interesting. In this case, it makes sense for both SIZE and RSS to
increase by the allocated memory amount, as that memory is local to
the application and not on the video hardware.
However the question remains: Did xlock's SIZE or RSS increase by 128M
(due to the mmap() )?
P.S.: Don't mean to be a pain in the butt. I'm really curious now.
===
As I said before... (3.00 / 1) (#55)
by fluffy grue (#FF00FF @ trikuare . cx) on Tue Jan 9th, 2001 at
06:55:27 PM EST
(User Info) http://trikuare.cx
xlock's SIZE is 162M, and its RSS is 2928. Its SHARE is 2244. Its SWAP
is 0. It's at this size whether it's doing any mode, not just a
GL-using one, ostensibly because it's always loading libGL.
===