Web application (app) exploration is a crucial part of various analysis and testing techniques. However, the current methods are not able to properly explore the state space of web apps. As a result, techniques must be developed to guide the exploration in order to get acceptable functionality coverage for web apps. Reinforcement Learning (RL) is a machine Learning method in which the best way to do a task is learned through trial and error, with the help of positive or negative rewards, instead of direct supervision. Deep RL is a recent expansion of RL that makes use of neural networks’ Learning capabilities. This feature makes Deep RL suitable for exploring the complex state space of web apps. However, current methods provide fundamental RL. In this research, we offer DeepEx, a Deep RL-based exploration strategy for systematically exploring web apps. Empirically evaluated on seven open-source web apps, DeepEx demonstrated a 17% improvement in code coverage and a 16% enhancement in navigational diversity over the stateof-the-art RL-based method. Additionally, it showed a 19% increase in structural diversity. These results confirm the superiority of Deep RL over traditional RL methods in web app exploration.

Web application  rewalls (WAFs) are used for protecting web applications from attacks such as SQL injection, cross-site request forgery, and cross-site scripting. As a result of the growing complexity of web attacks, WAFs need to be tested and updated on a regular basis. There are various tools and techniques to verify the correct performance of a WAF. But most of the techniques are manual or use brute-force attacks, so su er from poor e cacy. In this work, we propose a solution based on Reinforcement Learning (RL) to discover malicious payloads, which are able to bypass WAFs. We provide an RL framework with an environment compatible with OpenAI gym toolset standards. The environment is employed for training agents to implement WAF circumvention tasks. The agent mutates the syntax of a malicious payload using a set of modi cation operators as actions, without changes to its semantic. Then, upon WAF's reaction to the payload, the environment ascertains a reward for the agent. Eventually, based on these rewards, the agent learns a suitable sequence of mutations for any malicious payload. The payloads, which bypass the WAF determine rules defects, which can be further used in rule tuning for rule-based WAFs. Also, it can enrich the machine Learning-based WAFs datasets for retraining. We use Q-Learning, Advantage Actor-Critic (A2C), and Proximal Policy Optimization (PPO) algorithms with the Deep neural network. Our solution is successful in evading signature-based and machine Learning-based WAFs. While our focus in this work is on SQL injection, the method can be simply extended to use for any string-based injection attacks.

The hybrid electric train which operates without overhead wires or traditional power sources relies on hydrogen fuel cells and batteries for power. These fuel cell-based hybrid electric trains (FCHETs) are more efficient than those powered by diesel or electricity because they do not produce any tailpipe emissions making them an eco-friendly mode of transport. The target of this paper is to propose low-budget FCHETs that prioritize energy efficiency to reduce operating costs and minimize their impact on the environment. To this end, an energy management strategy [EMS] has been developed that optimizes the distribution of energy to reduce the amount of hydrogen required to power the train. The EMS achieves this by balancing battery charging and discharging. To enhance the performance of the EMS, proposes to use of a Deep Reinforcement Learning (DRL) algorithm specifically the Deep deterministic policy gradient (DDPG) combined with transfer Learning (TL) which can improve the system's efficiency when driving cycles are changed. DRL-based strategies are commonly used in energy management and they suffer from unstable convergence, slow Learning speed, and insufficient constraint capability. To address these limitations, an action masking technique to stop the DDPG-based approach from producing incorrect actions that go against the system's physical limits and prevent them from being generated is proposed. The DDPG+TL agent consumes up to 3. 9% less energy than conventional rule-based EMS while maintaining the battery's charge level within a predetermined range. The results show that DDPG+TL can sustain battery charge at minimal hydrogen consumption with minimal training time for the agent.