Linode Forum
Linode Community Forums 		 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic

Board index » Linux Community Forums » General Discussion
  Previous topic | Next topic 
Author Message
jbpros
PostPosted: Fri Jan 07, 2005 4:37 am 
Offline
Senior Newbie

Joined: Thu Dec 23, 2004 7:28 pm
Posts: 11
Website: http://www.jbpros.com/
Location: Belgium
Hi everybody!

For days I'm trying to compile mysql on my fresh gentoo installation but it still fails. It's a linode64 using 256mB of swap space plus 256 mB of swap in a file. oom-killer is still killing cc1plus with "all" that memory available. I don't understand well the informations output by dmesg about oom-killer. What mean the numbers about the swap space there?

Can I do something else than upgrading my linode to more RAM to succeed mysql compilation?

Note there is only gkrellmd and sshd daemons running.

Thank you!

Code:
$ emerge -av mysql
(...)
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_select.o -MD -MP -MF ".deps/sql_select.Tpo" \
  -c -o sql_select.o `test -f 'sql_select.cc' || echo './'`sql_select.cc; \
then mv -f ".deps/sql_select.Tpo" ".deps/sql_select.Po"; \
else rm -f ".deps/sql_select.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_do.o -MD -MP -MF ".deps/sql_do.Tpo" \
  -c -o sql_do.o `test -f 'sql_do.cc' || echo './'`sql_do.cc; \
then mv -f ".deps/sql_do.Tpo" ".deps/sql_do.Po"; \
else rm -f ".deps/sql_do.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_show.o -MD -MP -MF ".deps/sql_show.Tpo" \
  -c -o sql_show.o `test -f 'sql_show.cc' || echo './'`sql_show.cc; \
then mv -f ".deps/sql_show.Tpo" ".deps/sql_show.Po"; \
else rm -f ".deps/sql_show.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_string.o -MD -MP -MF ".deps/sql_string.Tpo" \
  -c -o sql_string.o `test -f 'sql_string.cc' || echo './'`sql_string.cc; \
then mv -f ".deps/sql_string.Tpo" ".deps/sql_string.Po"; \
else rm -f ".deps/sql_string.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_table.o -MD -MP -MF ".deps/sql_table.Tpo" \
  -c -o sql_table.o `test -f 'sql_table.cc' || echo './'`sql_table.cc; \
then mv -f ".deps/sql_table.Tpo" ".deps/sql_table.Po"; \
else rm -f ".deps/sql_table.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_test.o -MD -MP -MF ".deps/sql_test.Tpo" \
  -c -o sql_test.o `test -f 'sql_test.cc' || echo './'`sql_test.cc; \
then mv -f ".deps/sql_test.Tpo" ".deps/sql_test.Po"; \
else rm -f ".deps/sql_test.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_udf.o -MD -MP -MF ".deps/sql_udf.Tpo"\
  -c -o sql_udf.o `test -f 'sql_udf.cc' || echo './'`sql_udf.cc; \
then mv -f ".deps/sql_udf.Tpo" ".deps/sql_udf.Po"; \
else rm -f ".deps/sql_udf.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_update.o -MD -MP -MF ".deps/sql_update.Tpo" \
  -c -o sql_update.o `test -f 'sql_update.cc' || echo './'`sql_update.cc; \
then mv -f ".deps/sql_update.Tpo" ".deps/sql_update.Po"; \
else rm -f ".deps/sql_update.Tpo"; exit 1; \
fi
if g++ -DEMBEDDED_LIBRARY -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr\"" -DDATADIR="\"/var/lib/mysql\"" -DSHAREDIR="\"/usr/share/mysql\"" -I. -I. -I.. -I../bdb/build_unix -I./../include -I../include -I./.. -I.. -I.. -I../sql -I../regex    -O3 -DDBUG_OFF -O2 -march=pentium4 -fomit-frame-pointer -pipe -felide-constructors -fno-exceptions -fno-rtti   -fno-implicit-templates -fno-exceptions -fno-rtti -MT sql_yacc.o -MD -MP -MF ".deps/sql_yacc.Tpo" \
  -c -o sql_yacc.o `test -f 'sql_yacc.cc' || echo './'`sql_yacc.cc; \
then mv -f ".deps/sql_yacc.Tpo" ".deps/sql_yacc.Po"; \
else rm -f ".deps/sql_yacc.Tpo"; exit 1; \
fi
g++: Internal error: Terminated (program cc1plus)
Please submit a full bug report.
See <URL:http://bugs.gentoo.org/> for instructions.
make[3]: *** [sql_yacc.o] Error 1
make[3]: Leaving directory `/var/tmp/portage/mysql-4.0.22/work/mysql-4.0.22/libmysqld'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/var/tmp/portage/mysql-4.0.22/work/mysql-4.0.22/libmysqld'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/var/tmp/portage/mysql-4.0.22/work/mysql-4.0.22'
make: *** [all] Error 2

!!! ERROR: dev-db/mysql-4.0.22 failed.
!!! Function src_compile, Line 160, Exitcode 2
!!! compile problem
!!! If you need support, post the topmost build error, NOT this status message.


Code:
$ dmesg
(...)
Free pages:         256kB (0kB HighMem)
Active:12141 inactive:407 dirty:0 writeback:0 unstable:0 free:64 slab:1842 mapped:11757 pagetables:109
DMA free:256kB min:256kB low:512kB high:768kB active:48564kB inactive:1628kB present:65536kB
protections[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 2*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 256kB
Normal: empty
HighMem: empty
Swap cache: add 10502, delete 10360, find 2921/3493, race 0+0
Out of Memory: Killed process 18555 (cc1plus).
oom-killer: gfp_mask=0x1d2
DMA per-cpu:
cpu 0 hot: low 8, high 24, batch 4
cpu 0 cold: low 0, high 8, batch 4
Normal per-cpu: empty
HighMem per-cpu: empty

Free pages:         248kB (0kB HighMem)
Active:12298 inactive:225 dirty:0 writeback:0 unstable:0 free:62 slab:1846 mapped:11732 pagetables:126
DMA free:248kB min:256kB low:512kB high:768kB active:49192kB inactive:900kB present:65536kB
protections[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
DMA: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 248kB
Normal: empty
HighMem: empty
Swap cache: add 16477, delete 16414, find 5536/6541, race 0+0
Out of Memory: Killed process 19055 (sshd).
oom-killer: gfp_mask=0xd2
DMA per-cpu:
cpu 0 hot: low 8, high 24, batch 4
cpu 0 cold: low 0, high 8, batch 4
Normal per-cpu: empty
HighMem per-cpu: empty

Free pages:         744kB (0kB HighMem)
Active:12465 inactive:0 dirty:0 writeback:0 unstable:0 free:186 slab:1783 mapped:11705 pagetables:113
DMA free:744kB min:256kB low:512kB high:768kB active:49860kB inactive:0kB present:65536kB
protections[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
DMA: 92*4kB 13*8kB 3*16kB 1*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 744kB
Normal: empty
HighMem: empty
Swap cache: add 28538, delete 25596, find 7949/10099, race 0+0
Out of Memory: Killed process 11811 (cc1plus).
oom-killer: gfp_mask=0xd2
DMA per-cpu:
cpu 0 hot: low 8, high 24, batch 4
cpu 0 cold: low 0, high 8, batch 4
Normal per-cpu: empty
HighMem per-cpu: empty

Free pages:         584kB (0kB HighMem)
Active:11833 inactive:633 dirty:0 writeback:0 unstable:0 free:146 slab:1821 mapped:11706 pagetables:115
DMA free:584kB min:256kB low:512kB high:768kB active:47332kB inactive:2532kB present:65536kB
protections[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
DMA: 34*4kB 14*8kB 7*16kB 1*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 584kB
Normal: empty
HighMem: empty
Swap cache: add 44798, delete 41944, find 12964/16526, race 0+0
Out of Memory: Killed process 2434 (cc1plus).
Top
   
Reply with quote  
pclissold
 Post subject:
PostPosted: Fri Jan 07, 2005 5:25 am 
Offline
Senior Member
User avatar

Joined: Fri Oct 24, 2003 3:51 pm
Posts: 965
Location: Netherlands
Are you trying to do anything unusual with MySQL? I have compled it under Gentoo on a 64M Linode without encountering this.

Post the output of
Code:
cat /proc/io_status
and
Code:
vmstat -s
so we can see if anything is broken.

_________________
/ Peter
Top
   
Reply with quote  
jbpros
 Post subject:
PostPosted: Fri Jan 07, 2005 8:45 pm 
Offline
Senior Newbie

Joined: Thu Dec 23, 2004 7:28 pm
Posts: 11
Website: http://www.jbpros.com/
Location: Belgium
Code:
nyat root # cat /proc/io_status
io_count=606999 io_rate=0 io_tokens=399995 token_refill=512 token_max=400000


Code:
nyat root # vmstat -s
        59512  total memory
        58376  used memory
        11132  active memory
        27820  inactive memory
         1136  free memory
        27652  buffer memory
         8588  swap cache
       528572  total swap
         2600  used swap
       525972  free swap
       254995 non-nice user cpu ticks
         3412 nice user cpu ticks
      1225387 system cpu ticks
     15639522 idle cpu ticks
       113930 IO-wait cpu ticks
           28 IRQ cpu ticks
        23613 softirq cpu ticks
      1212233 pages paged in
      1215856 pages paged out
        24937 pages swapped in
        23496 pages swapped out
     18405981 interrupts
      3062915 CPU context switches
   1104972404 boot time
       231341 forks

some hours after the last mysql compilation attempt (without reboot). There is nothing strange to me there..
Top
   
Reply with quote  
jbpros
 Post subject:
PostPosted: Fri Jan 07, 2005 8:50 pm 
Offline
Senior Newbie

Joined: Thu Dec 23, 2004 7:28 pm
Posts: 11
Website: http://www.jbpros.com/
Location: Belgium
I've been using the -march=pentium4 flag in /etc/make.conf. I read old stuffs about this flag being problematic. Any comment?
Top
   
Reply with quote  
Jay
 Post subject:
PostPosted: Sat Jan 08, 2005 12:12 am 
Offline
Senior Member

Joined: Sun Nov 14, 2004 6:37 pm
Posts: 138
Website: http://oldos.org
WLM: jasonlfaulkner@hotmail.com
Yahoo Messenger: jasonfncsu
AOL: jaylfaulkner
Location: NC, USA
Emerge info == ?

_________________
Jay Faulkner
http://oldos.org
Top
   
Reply with quote  
pclissold
 Post subject:
PostPosted: Sat Jan 08, 2005 5:40 am 
Offline
Senior Member
User avatar

Joined: Fri Oct 24, 2003 3:51 pm
Posts: 965
Location: Netherlands
jbpros wrote:
some hours after the last mysql compilation attempt (without reboot). There is nothing strange to me there..

Your io_status and vmstat do indeed look normal.


jbpros wrote:
I've been using the -march=pentium4 flag in /etc/make.conf. I read old stuffs about this flag being problematic. Any comment?

My Linode is a Gentoo stage one install with
Code:
CHOST="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"
and it has compiled MySQL OK

_________________
/ Peter
Top
   
Reply with quote  
jbpros
 Post subject:
PostPosted: Sat Jan 08, 2005 8:00 am 
Offline
Senior Newbie

Joined: Thu Dec 23, 2004 7:28 pm
Posts: 11
Website: http://www.jbpros.com/
Location: Belgium
Code:
nyat root # emerge info
Portage 2.0.51-r3 (default-linux/x86/2004.3, gcc-3.3.4, glibc-2.3.4.20040808-r1, 2.6.9-linode9 i686)
=================================================================
System uname: 2.6.9-linode9 i686 UML
Gentoo Base System version 1.4.16
ccache version 2.3 [enabled]
Autoconf: sys-devel/autoconf-2.59-r5
Automake: sys-devel/automake-1.8.5-r1
Binutils: sys-devel/binutils-2.15.90.0.1.1-r3
Headers:  sys-kernel/linux26-headers-2.6.8.1-r1
Libtools: sys-devel/libtool-1.5.2-r7
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo http://prometheus.cs.wmich.edu/gentoo http://mirror.datapipe.net/gentoo"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"
USE="x86 acl apache2 apm arts avi berkdb bitmap-fonts cgi crypt cups dba encode foomaticdb fortran freetype gd gdbm gif imap ipv6 jabber jpeg libwww mad maildir md5sum mikmod mime motif mpeg msn mysql ncurses nls nptlonly oggvorbis oscar oss pam pdflib perl php png ppds python quicktime readline sdl spell ssl svga tcpd truetype xml2 xmms yahoo zlib"


Code:
nyat root # emerge -av mysql;

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild  N    ] dev-db/mysql-4.0.22  +berkdb -debug -innodb +perl +readline (-selinux) +ssl -static +tcpd 0 kB
[ebuild  N    ] dev-perl/DBD-mysql-2.1027  0 kB

Total size of downloads: 0 kB


Code:
nyat root # cat /etc/make.conf
# These settings were set by the catalyst build script that automatically built this stage

# --- Host-specific flags & Use flags ---
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
USE="apache2 cgi dba freetype gd -gnome -gpm -gtk -gtk2 imap jabber -kde maildir md5sum mime msn mysql nptlonly oggvorbis oscar php ppds -qt yahoo -opengl -xpm -xv -X"

# --- Compilation flags ---
MAKEOPTS="-j1"
FEATURES="ccache"
PORTAGE_NICENESS="0"

# --- Directories ---
PORTDIR="/usr/portage"
DISTDIR="${PORTDIR}/distfiles"
PORTDIR_OVERLAY="/usr/local/portage"
DISTCC_DIR="/var/tmp/portage/.distcc"

# --- Mirrors ---
GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo http://prometheus.cs.wmich.edu/gentoo http://mirror.datapipe.net/gentoo"


SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"


Note: I'm now trying again the compilation without ccache.

I just noticed the "nptlonly" flag being there. Does it have some effect when nptl is off? I think not but maybe I'm wrong?

Do you notice something else wrong?

Thank you for your kind help.
Top
   
Reply with quote  
jbpros
 Post subject:
PostPosted: Sat Jan 08, 2005 9:21 am 
Offline
Senior Newbie

Joined: Thu Dec 23, 2004 7:28 pm
Posts: 11
Website: http://www.jbpros.com/
Location: Belgium
I found this:

http://mysqld.active-venture.com/Compilation_problems.html

Quote:
If you get errors when compiling `sql_yacc.cc', such as the ones shown here, you have probably run out of memory or swap space:

Internal compiler error: program cc1plus got fatal signal 11
or
Out of virtual memory
or
Virtual memory exhausted

The problem is that gcc requires huge amounts of memory to compile `sql_yacc.cc' with inline functions. Try running configure with the --with-low-memory option:

shell> ./configure --with-low-memory

This option causes -fno-inline to be added to the compile line if you are using gcc and -O0 if you are using something else. You should try the --with-low-memory option even if you have so much memory and swap space that you think you can't possibly have run out. This problem has been observed to occur even on systems with generous hardware configurations, and the --with-low-memory option usually fixes it.


I thus added -fno-inline to my CFLAGS. It's compiling now, results in a while.
Top
   
Reply with quote  
jbpros
 Post subject:
PostPosted: Sat Jan 08, 2005 10:50 am 
Offline
Senior Newbie

Joined: Thu Dec 23, 2004 7:28 pm
Posts: 11
Website: http://www.jbpros.com/
Location: Belgium
It worked!

Anyway I'm still not convinced that 512mb of swap + 64mb of RAM would have caused OOM situations when compiling mysql...

Thanks for your help.
Top
   
Reply with quote  
SteveG
 Post subject:
PostPosted: Mon Jan 10, 2005 12:31 pm 
Offline
Senior Member

Joined: Sun Nov 30, 2003 2:28 pm
Posts: 245
The OOM killer can be activated long before you run out of swap. I don't remember the details, and searching for "OOM killer" in the kernel mailling list returns *far* too many hits, but it has to do with interaction of process VM with the I/O buffers. Since this happens while you're compiling, which does lots of I/O, I'm guessing that this is what you're hitting.

_________________
The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world.
-- seen on the net
Top
   
Reply with quote  
Display posts from previous:  Sort by  
Post new topic  Reply to topic

Board index » Linux Community Forums » General Discussion


Who is online

Users browsing this forum: dobriain and 1 guest

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group