Op-Ed: Outpacing AI-driven cyber attacks

Australian organisations are expected to spend almost $6.2 billion on information security and risk management products and services in 2025, up 14.4 per cent from 2024 (Gartner) – a clear indication of the critical importance of cyber security in protecting digital assets and maintaining business continuity in an increasingly digital world.

Richard Addiscott from Gartner notes that “AI will significantly impact security strategies, requiring leaders to adapt to evolving threats, talent gaps and regulatory challenges.” As such, security leaders must understand AI’s role in the threat landscape to develop intelligent, adaptive strategies and stay ahead of sophisticated attacks.

Understanding identity attacks and the role of cloud

AI tools are not only enabling attackers to create malicious code but also accelerating their entire attack process. Attackers can now gather information with automated reconnaissance and bypass protective measures much faster than before. This is evident in identity attacks and cross-border AI risks.

VIEW ALL

Push Security research highlights a new method of bypassing identity authentication controls – cross-IDP impersonation. This method allows attackers to exploit SaaS applications and identity providers (IDPs), impersonating users without triggering security alerts.

This issue is particularly prevalent in cloud environments, where attackers can mimic domains, bypassing multi-factor authentication (MFA), and access sensitive systems undetected. These attacks are incredibly challenging to detect, with traditional and even modern CNAP security tools often failing to pick up, leaving organisations unaware that a breach has occurred.

This underscores the need for robust identity and access management systems, especially for cloud-reliant organisations. As defenders, we must stay vigilant and continually reassess our security controls to address emerging threats. The more cloud-based a company is, the greater the risk, making it crucial to adapt and strengthen security measures continuously.

The growing concern of cross-border AI risks

Another issue leaders must be aware of is the risks associated with cross-border data and AI tool usage. As Generative AI (GenAI) integrates into business operations, concerns about data breaches and balancing regulatory demands and ethical use grow.

Governments, especially in the EU, are drafting AI regulations, with other countries such as the US, Australia, and New Zealand likely to follow. The challenge is managing data jurisdiction and residency, as AI tools often work across borders without clear boundaries.

Organisations need controls like data masking and policies that safeguard data in the correct regions. However, regulations alone won’t solve this problem. Education and awareness are essential for ensuring the secure management of these tools by organisations and their employees.

The rise of shadow AI and what it means for organisations

In a similar vein, the rise of GenAI tools like DeepSeek AI also introduces the unique challenge of ‘shadow AI’. Many companies are grappling with how to manage these tools effectively, given the risks they pose. While banning these tools outright might seem like a solution, it often leads to their ‘shadow AI’ use without the organisation’s knowledge.

We saw this play out in Australia, where GenAI tools have prompted a rethink on safe and ethical usage. The challenge lies not just in the technology itself but in how people use it. Security and business leaders must educate staff on the benefits and risks of GenAI, similar to how education on phishing has been approached in the past – people need to be made aware of the risks and how to use them responsibly, rather than just issuing a blanket ban.

Beyond awareness, organisations must be proactive in developing secure AI frameworks to ensure data protection, retention and safe sharing.

Understanding the threat landscape: What CISOs need to know

Given the rise of AI-powered attacks, it’s crucial for Chief Information Security Officers (CISOs) to adapt their strategies accordingly. The key to defending against these threats is understanding how attackers operate. AI is just another tool that attackers use to speed up their efforts. If your mean-time-to-remediation (MTTR) is still sitting in days, you’ve got a problem.

The reality is that attackers are still after the same thing – data or disruption. AI doesn’t change their goal – it just brings the goalpost closer. Identifying threats early and stopping them before they cause harm puts you in the strongest position possible to defend your goal line.

One of the best ways to get ahead of these threats is by focusing on early detection. Continuous testing of a security environment is a powerful starting point – whether through vulnerability scanning, red teaming or simulated attacks. It’s about identifying attack vectors and closing them as quickly as possible. The faster you shut down attack paths, the better your security posture will improve.

The future of AI in cybersecurity: Automation and ethical considerations

AI-driven automation will increasingly enhance threat detection and response, but many organisations hesitate to fully adopt it due to concerns about errors and disruptions. Automated responses can be powerful, but risk stopping business-critical processes or triggering false positives.

GenAI has an incredible potential to improve decision-making and as AI becomes more sophisticated, automated threat responses will be more reliable. But building trust in AI systems and ensuring they don’t disrupt essential business operations will take time but it needs to be undertaken.

Undeniably, the intersection of AI and cyber security is reshaping defense strategies, offering both opportunities and risks. By focusing on education, collaboration, continuous testing and early detection, organisations can strengthen defences against the evolving threat landscape.