Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
 Post subject: Excessive transfer usage
PostPosted: Fri Jun 25, 2004 3:32 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2004 4:45 pm
Posts: 19
Website: http://jeffmacmichael.com
Location: Maryland, USA
Hi gang... Linode n00b here. I'm psyched to get it up and running and have true control over my hosting. This looks like a great spot to do it. <g>

First question:

I just got a 64 MB Linode yesterday, and in just about 24 hours, I've seemed to generate...

Incoming: 1.55 GB
Outgoing: 1.34 GB
Total: 2.90 GB

...of transfer. It's 12% of my monthly allowance. All I've done is set a Gentoo install, trash it, and set up a Debian install. And get SSH working... didn't start using the server yet or cofigure any services. How on earth could I have used up all that bandwidth? Or am I getting that many port scans...? ;)

/puzzled :?

ged


Top
   
 Post subject:
PostPosted: Fri Jun 25, 2004 3:38 pm 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3090
Website: http://www.linode.com/
Location: Galloway, NJ
"iptraf" and "tcpdump" are your friends :)

BTW, for Debian: apt-get install iptraf tcpdump

-Chris


Top
   
 Post subject:
PostPosted: Sat Jun 26, 2004 10:49 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2004 4:45 pm
Posts: 19
Website: http://jeffmacmichael.com
Location: Maryland, USA
Thanks caker. Turning on the firewall is a good idea as well. <g> I'm still having a lot of my bandwidth burned up though.

I did a quick dump and found that most of the traffic that's hitting me is requests to open the IRC port. I don't have an ircd and the port's closed, but I'm still getting a ton of them. I kind of expect port scans but I wasn't expecting that.

http://gednet.com/ip_traffic-2.log

Anyone have an idea what I could do?


Top
   
 Post subject:
PostPosted: Sun Jun 27, 2004 1:34 am 
Offline
Senior Newbie

Joined: Mon Jun 21, 2004 9:55 pm
Posts: 19
Website: http://kiwiinjection.com
ged wrote:
I just got a 64 MB Linode yesterday, and in just about 24 hours, I've seemed to generate...

Incoming: 1.55 GB
Outgoing: 1.34 GB
Total: 2.90 GB

...of transfer.
ged wrote:
Thanks caker. Turning on the firewall is a good idea as well. <g> I'm still having a lot of my bandwidth burned up though.

I did a quick dump and found that most of the traffic that's hitting me is requests to open the IRC port. I don't have an ircd and the port's closed, but I'm still getting a ton of them. I kind of expect port scans but I wasn't expecting that.

http://gednet.com/ip_traffic-2.log

Anyone have an idea what I could do?

The packets coming in on 6667 are one thing, but if they're not connecting, how much can they total? Looking at the log, the packets to 6667 are 48 bytes each.

What is going out in 24 hours that added up to 1.34 GB? grep the log for the outgoing.

About the incoming: if you did two installs, that's likely to be a big chunk of your total incoming (1.55 GB), no? Gentoo is 800 MB itself.


Top
   
 Post subject:
PostPosted: Sun Jun 27, 2004 8:00 am 
Offline
Senior Newbie

Joined: Thu Jun 24, 2004 4:45 pm
Posts: 19
Website: http://jeffmacmichael.com
Location: Maryland, USA
fieschko wrote:
The packets coming in on 6667 are one thing, but if they're not connecting, how much can they total? Looking at the log, the packets to 6667 are 48 bytes each.

What is going out in 24 hours that added up to 1.34 GB? grep the log for the outgoing.

Good point, the numbers just don't add up. I'll run a trace for a while and try to see what's what.

fieschko wrote:
About the incoming: if you did two installs, that's likely to be a big chunk of your total incoming (1.55 GB), no? Gentoo is 800 MB itself.
Also a good point; that's the first thing I thought of. I did a Gentoo install @ 800 MB and then a Debian at 80 MB, plus maybe about ~120mb worth of installs to the deb box (apache, php, etc.)

I'll run some dumps for a while.


Top
   
 Post subject:
PostPosted: Sun Jun 27, 2004 9:26 am 
Offline
Senior Newbie

Joined: Mon Jun 21, 2004 9:55 pm
Posts: 19
Website: http://kiwiinjection.com
ged wrote:
I'll run a trace for a while and try to see what's what.

I did a Gentoo install @ 800 MB and then a Debian at 80 MB, plus maybe about ~120mb worth of installs to the deb box (apache, php, etc.)

I'll run some dumps for a while.

Or, watch in realtime: do `iptraf -f` in an ssh session, select 'Statistical breakdown -- by port'


Top
   
 Post subject:
PostPosted: Sun Jun 27, 2004 9:50 am 
Offline
Senior Member

Joined: Wed Aug 13, 2003 10:24 am
Posts: 55
It seems like there is a lot of UDP traffic between your linode and host9.linode.com. Is that your host?
Code:
Sat Jun 26 22:29:58 2004; UDP; eth0; 208 bytes; from host9.linode.com:53 to li4-153.members.linode.com:2060
Sat Jun 26 22:29:58 2004; UDP; eth0; 71 bytes; from li4-153.members.linode.com:2061 to host9.linode.com:53

Transfers within the datacenter aren't supposed to count against your bandwidth so I'm not sure if this is the primary issue, but I looked at 1 second of your log and of 104 packets, 58 were to or from host9.


Top
   
 Post subject:
PostPosted: Sun Jun 27, 2004 11:04 am 
Offline
Senior Member
User avatar

Joined: Mon Jun 23, 2003 1:25 pm
Posts: 260
Data transfer that does not go past the switch but I think there is a current bug in the accounting of transfer that if the transfer is across subnets then the transfer is added to the montly transfer.

The data from the host looks like DNS since the host port is 53

After taking a look at the IPs for the linode and the host they do appear to be on different subnets. So that may be where some of the bandwidth usage is coming from

Adam


Top
   
 Post subject:
PostPosted: Sun Jun 27, 2004 10:58 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2004 4:45 pm
Posts: 19
Website: http://jeffmacmichael.com
Location: Maryland, USA
Well I realized that I left name lookup on when I ran iptraf & tcpdump. That certainly didn't help... :roll:

I've run another one w/name lookup off and hope to get something more meaningful out of it.


Top
   
 Post subject:
PostPosted: Mon Jun 28, 2004 2:39 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2004 4:45 pm
Posts: 19
Website: http://jeffmacmichael.com
Location: Maryland, USA
fieschko wrote:
Or, watch in realtime: do `iptraf -f` in an ssh session, select 'Statistical breakdown -- by port'

Done. I ran stats by port for about 40 minutes today.

I'm at a loss... I had about 1.7 MB/min of IRC connect attempts. If that's average, then that would add up to about 2.4 GB per day (72 GB per month!) in transfer. Wheeeeeeee :D The non-irc stuff seems negligible.

If anyone is curious (or thinks I'm nuts), here's the stats dump [.zip format], and here's the top 20 list [html, small].

Must've been an irc service on this IP before I guess.

It's all 48 byte requests, with a 40 byte reply back from my machine. Even if I figure out how to not reply (I'm still learning) I'm going to be looking at about a GB of traffic a day.

I'm assuming that out of the box Debian won't act as any kind of irc relay, right? I don't know squat about ircd.

Chris - any ideas? Would Linode be willing to give me a different IP? Assuming folks out there notice they're not connecting, I hope it should die down in a few weeks (?) and the IP would be useable again.

ged


Top
   
 Post subject:
PostPosted: Mon Jun 28, 2004 3:08 pm 
Offline
Senior Member
User avatar

Joined: Mon Jun 23, 2003 1:25 pm
Posts: 260
I see no reason why caker would not be willing to give you a new IP. As it is nothing you have encouraged.

Adam


Top
   
 Post subject:
PostPosted: Mon Jun 28, 2004 3:35 pm 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3090
Website: http://www.linode.com/
Location: Galloway, NJ
ged wrote:
Chris - any ideas? Would Linode be willing to give me a different IP? Assuming folks out there notice they're not connecting, I hope it should die down in a few weeks (?) and the IP would be useable again.

Absolutely. Shoot me a support ticket. I think this IP was mentioned by someone else as having excessive traffic, so perhaps I'll quarantine it for a while.

-Chris


Top
   
 Post subject:
PostPosted: Mon Jun 28, 2004 4:57 pm 
Offline
Senior Newbie

Joined: Mon Jun 21, 2004 9:55 pm
Posts: 19
Website: http://kiwiinjection.com
ged wrote:
It's all 48 byte requests, with a 40 byte reply back from my machine. Even if I figure out how to not reply (I'm still learning) I'm going to be looking at about a GB of traffic a day.
ged

With portsentry it's simple to create an iptables INPUT rule to DROP on 6667, log the offending ip and then to ignore all connection attempts from that ip.


Top
   
 Post subject:
PostPosted: Mon Jun 28, 2004 6:44 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2004 4:45 pm
Posts: 19
Website: http://jeffmacmichael.com
Location: Maryland, USA
caker wrote:
Shoot me a support ticket.

Done!

fieschko wrote:
With portsentry it's simple to create an iptables INPUT rule to DROP on 6667, log the offending ip and then to ignore all connection attempts from that ip.

Thanks for the tip fieschko... I'll check it out.

Thanks for the support everyone :mrgreen:


Top
   
 Post subject:
PostPosted: Fri Aug 06, 2004 7:08 am 
Offline
Junior Member

Joined: Thu May 13, 2004 8:08 am
Posts: 27
caker wrote:
"iptraf" and "tcpdump" are your friends :)


ntop is much better, in regards that it reports to you via a web interface. there are a lot of reports, you can even drill down to specific protocol (pop3, http, dns, etc). I can't believe I never heard this awesome utility before.... :oops:

For debian users, it's apt-get install ntop, then you can see the report straight away via http://yourserver:3000
Nifty.

edit:
see mine here - http://ntop.abangadek.com/


cheers, HS


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: wolfsoft and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group