Three New arXiv Papers Address ML Model Architecture and Security
Three recent papers published on arXiv explore different aspects of machine learning model design and vulnerabilities.
Stochastic Activations in Language Models
According to arXiv paper 2509.22358v2, researchers have introduced “stochastic activations,” a novel approach that randomly selects between multiple non-linear functions in the feed-forward layer of large language models. The paper specifically describes choosing between SILU or RELU activation functions based on a Bernoulli draw.
Membership Inference Attacks with Partial Features
ArXiv paper 2508.06244v2 addresses privacy concerns in machine learning models, noting that “machine learning models are vulnerable to membership inference attack, which can be used to determine whether a given sample appears in the training data.” The paper states that most existing methods assume attackers have full access to features of target data.
Limitations of Mask Diffusion Models
A third paper (arXiv:2510.03289v2) examines why mask diffusion approaches face challenges. According to the abstract, diffusion language models offer advantages over autoregressive models through “their ability to support parallel generation and bidirectional attention, enabling a more controllable generation process.”
All three papers represent cross-listed submissions to the cs.AI category on arXiv.