* | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, “.*400\s+(?!test).*")
This will look for all messages you collected (that's the * at the beginning of the line) and then filter those to only messages that contain “400” that are NOT followed by “test”, assuming there is at least one space character in between.
Let me explain how this works. The * at the beginning says "find all messages". The result of that is then piped (that's the | character) to the extract search command. The extract search command will extract data from the field you specify based on the regex you enter. In this example, I told it to extract everything in the raw message (that's the _raw field). The regex there, "(?<MSGS>.*)", says to create a new field (also known in the regex world as a capturing group) named "MSGS" from the _raw field that matches the regex .*. Well, as you likely know, .* means zero or more of any character - which basically means everything. So it returns the entire raw message and assigns it to a new field called MSGS.
Why did I do that? Because I need to use the filter search command in order to do the negative search you are looking for - where the word "test" does not exist.
So I pipe the output of the extract search command to the filter search command. I use the regex option of the filter command so I can specify a negative lookahead regex. This regex says match anything, followed by 400, followed by one or more spaces, followed by anything except the word "test", followed by anything.
If in your case you have the word "test" appear before the number 400, then your search command would be:
* | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, “.*test\s+(?!400).*")
I hope that make sense. Let me know if you need anything else!
Thanks Scott for your response.
I tried above regex, but I am not seeing the expected result. Not sure if I am doing something wrong.
I used below query
COLLECTOR_NAME="Coll" | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, ".*400\s+(?!test).*")
I have uploaded below lines in the above collector (Coll), so the pipe should send below lines to the next extract command....
After executing above query, it still outputs all the lines.
Mar 28, 2018 9:31:55 AM
400 with username, ?invalid_grant Bad credentials" ? error description then do not create any alert.
Mar 28, 2018 9:31:55 AM
400 Hi this is test result
Mar 28, 2018 9:31:55 AM
 : If you get 400 with ?unauthorized Cannot find user with uid? error description
I'm glad you sent me a few examples. I have fixed the search query. To match those lines properly, you would run the following query:
COLLECTOR_NAME="Coll" | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, ".*400[\]\w]+(?!test).*")
The difference is in the regex for the filter search command. First, I assumed that there was a space character after the 400. That's why I had "400\s+" originally. In your case, you had two different characters - a tab (I think), and a "]". So, I updated the regex to handle these with "400[\]\w]+". This means 400, followed by one or more of either a white space (any non-visible, or whitespace character) or a "]". If you have more lines in your actual source log file that may have other characters after the 400 - something like a colon ( : ), semi-colon ( ; ), dash ( - ), etc. - then you need to add those inside the square brackets of the regex I mentioned above. And you will need to escape those characters that are regex special characters, if any.
Please let me know if you need any more help.
COLLECTOR_NAME="SysLog_TCP_XXX" && "Http Status=" | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, ".*(?!Bad credentials).*")
I was trying above and I still see records containing "Bad credential" being returned by the query. I want the result with http status 400 which does not contain string "Bad credentails" anywhere in it .
Can you paste a few example lines from your logs that have what you are looking for and what you want excluded so I can do some debugging? I'm guessing it's just something similar to what we saw before where you have to put in some additional regex to describe things like white spaces. In fact, I am going to guess right now that the following might work:
COLLECTOR_NAME="SysLog_TCP_XXX" && "Http Status=" | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, ".*(?!Bad[\]\w]+credentials).*")
Let me know.
1 of 1 people found this helpful
Thanks Scot for you help.
Below is the solution. If you see error related to double quotes, type them again.
COLLECTOR_NAME=“test" && "Http Status=" | extract field=_raw "(?<MSGS>.*)" | filter regex(MSGS, "^((?!Bad\s+credentials).)*$”)
Thanks for updating the community, Sameer. Yes, if you look close at the query you posted above, the last double-quote looks different - it's a different font. sometimes copy/paste gets funky...