using vim instead of sed
in order to process a text file in any sense, and being used with vim, and not with sed
i use the following for deleting googlebot lines from a web log.
vim -e -c ":d/googlebot/g" -c :wq website.YYYYMMDD.log
-e doesn’t put vim in interactive mode, so you can script it and put it in crontab.
or if you need to pipe it:
cat website.YYYYMMDD.log | vim - -e –c ":d/googlebot/g" -c :wq
maybe i will add more vim issues as comments to this post.