LLM As A Judge is not the shortcut you think