Enter GNU Parallel, a shell tool designed for executing tasks in parallel using one or more computers. For my purposes I just ran in on a single system, but wanted to take advantage of multiple cores.

Having enough memory on my system, I loaded the entire massive file into memory and pipe it to GNU Parallel along with another file consisting of thousands of different strings I want to search for in the “PATTERNFILE”:

cat BIGFILE | parallel –pipe grep -f PATTERNFILE

via Speeding Up Grep Log Queries with GNU Parallel – The State of Security.