Diagnose Before You Refine: Multi-Dimensional Judge Feedback for Query-Efficient Jailbreak Attacks

AI security research · arXiv cs.CR (AI security)

Diagnose Before You Refine: Multi-Dimensional Judge Feedback for Query-Efficient Jailbreak Attacks

REDCORE assessment

The research introduces JSB, a method that uses multi-dimensional feedback to optimize adversarial prompts with extreme query efficiency. It demonstrates that current industry-standard safety filters are vulnerable to low-query-count attacks. For a defender, this highlights that reliance on a single primary safety layer is insufficient against adaptive, automated adversarial strategies.

Impact for lean teams

Small teams using LLMs should expect that even basic safety filters can be bypassed with minimal effort by attackers. They should prioritize implementing robust input monitoring and rate limiting as these automated attacks are becoming increasingly efficient.

Source summary

Automated jailbreak attacks rely on evaluation feedback to optimize adversarial prompts, but existing methods often lack fine-grained diagnostic information. We propose Jev-Guided Strategy Bandit (JSB), a query-based jailbreak framework that leverages multi-dimensional probabilistic feedback to guide prompt optimization. JSB combines continuous jailbreak scoring with targeted refinement, strategy switching, and best-state reseeding to balance exploration and refinement. We evaluate JSB on two harmful request datasets across five open-source and three closed-source LLMs. Under a budget of only 10 target-model queries, JSB achieves an average HarmBench attack success rate of 89.8\% across ten open-source settings, consistently outperforming existing automated jailbreak attacks. Further analyses demonstrate its effectiveness under varying query budgets and attacker models. Defense experimen

Recommended actions

• Implement rate limiting on API endpoints to prevent high-frequency adversarial probing.
• Utilize secondary, independent moderation models to scan inputs for adversarial intent.
• Configure LLM parameters to reduce output temperature for tasks prone to exploitation.
• Establish automated anomaly detection to alert on repetitive or suspicious query patterns.

This assessment was generated by REDCORE AI from the attributed source above and is provided for situational awareness. Validate against the original research and your own environment before acting.