Python to parse fields in Amazon S3 logs

The log format for Amazon S3 is slightly annoying. Not overwhelmingly so, but the date field has the field separator (a space) in the middle of it and it isn’t encapsulated by quote characters. Here’s some code to split the fields up, assuming you’ve downloaded the log file already (it’s easy enough to list all logs and retrieve them with boto):

import csv
r = csv.reader(open('logfilename'),
        delimiter=' ',quotechar='"')
log_entries = []
for i in r:
    i[2] = i[2] + " " + i[3] # repair date field
    del i[3]
    log_entries.append(i)



0 comments ↓

There are no comments yet...Kick things off by filling out the form below..

Leave a Comment