Monitoring users bandwidth

Can people think of an effective way to monitor a users bandwidth?

I am running a site where a user will have a blog type site on a subdomain.

for example.

kalyse.domain.com

All this is an A DNS Record with a wildcard that forwards to a single script on a server. Theres no actual subdomaining involved.
The problem is I need to count the bandwidth a user uses for anything from their 'domain'. This includes any downloads or what not.

I cant really think of a way except log trawling... but surely this is going to be one pain in the ass. How do software panels do this effectively?

 

they count per subdomain

 

Best resonse ever. You have really englightened me.

Anywayyy. They dont 'count' per subdomain,there wouldnt be anything to count. They would have to trawl logs or something Imagine.

Does anyone have any useful feedback that I may be able to use? Thanks

 

<?
function cpanel_logs( $username, $password, $domain, $searchpath )
{
	$bw = 0 ;
	$logs = file( sprintf(  'http://%s:%s@%s:2082/getlogarchive/%s', 
							$username, 
							$password, 
							$domain, 
							$domain ));
	if( $logs )
	{
		foreach( $logs as $line )
		{
			preg_match( '~"([A-Z]+) (.*?[^ ]) HTTP/1\.[1|0]" ([0-9]+) ([0-9]+) "(.*?[^"])" "(.*?[^"])"~si', 
						$line, 
						$matches 
			);
			
			if( ereg( $searchpath, $matches[2] ) )
			{
				$bw += $matches[4];		
			}
		}
	}
	
	return $bw ;
}

$username = 'username';
$password = 'password';
$hostname = 'hostname.com';

printf( 'Images on %s have taken up a total of %01.2fMB bandwidth today<br />',
		$hostname,
		cpanel_logs( $username, $password, $hostname, 'images' ) / 1048576 
);
		
printf( '%s has taken up a total of %01.2fMB bandwidth today<br />',
		$hostname,
		cpanel_logs( $username, $password, $hostname, '/' ) / 1048576 
);
?>

PHP:

Is how I would do it with a cpanel server, that prolly won't work for you, you will need to match something else in the logs to identify where the traffic is going, but still that's how you would do it, load the log file into an array loop over each line matching the request location and status and numbers and then write some if clauses to collect the correct data from the matched data.

If you don't have access to cpanel, or still don't get it, then please take about 20 lines out of the log file you want to parse and post it, and tell me what is identifying each seperate "subdomain"

 

Ill grab the logs tomorrow. I cant get them right this second.

One problem or concern I have though, is CPU management. This project which Im designing is going to be pretty large. Ive already been working on the network for management and mysql pooling and balancing + high availability.

Im just concerned that because I have 5 apache servers running the seame script which the user is fed to depending on the load balancer, this could cause problems for updating, and the additional load on a server. Will parsing all the logs add considerably?

I dont know how much it would use, I havenever done anything with logs before but I can see this being a huge drain, especially if I am having to do this every hour, then I assume delete the logs or create a separate file for those logs that have been read.

 

I would do it every 24 hours when the system does, it will put some pressure on the server, but not uneven pressure, so long as your server isn't already overloaded theres no reason code like that ( no matter how long it runs ) would have an affect on performance.

 

I would look into log analysis as a last possibility, it will put significant load on the server in terms of I/O. Plus, with those regular expressions called for each line I would be seriously concerned on the stress the server would be under during this process. Hopefully your logs don't grow too fast.

I would imagine there would be some Apache modules available for monitoring bandwidth, I would be looking in that direction first.