Jump to content
Frequently Asked Questions
  • Are you not able to open the client? Try following our getting started guide
  • Still not working? Try downloading and running JarFix
  • Help! My bot doesn't do anything! Enable fresh start in client settings and restart the client
  • How to purchase with PayPal/OSRS/Crypto gold? You can purchase vouchers from other users
  • Neural Network Mouse Movement


    beezdul

    Recommended Posts

    Video

    This is a small project I'm working on where I use a neural network to create mouse movements. In theory, the movements themselves should be completely undetectable (and the results are pretty impressive and look almost exactly like my real movements). Currently I've trained it on about 4000 clicks from my own data.

     

    How it works:

    The network is a 3 layer fully connected network. The input is 6 float values (random value 0.000-0.001,last start delta X,last start delta Y,current delta X,current delta Y,1.0)

    The network has a 1000 neuron layer connected to a 400 neuron layer (trained with 50% dropout), and the activation function is leaky relu (tf.nn.leaky_relu).

    The outputs are 456 values, each ranging from -1 to 1. The output is grouped into 152 sets of 3, being the delta x, delta y, and either 1 or -1 (-1 signals that the movement is complete).

    The deltas are an amount from the end position, divided by 1000. (So if the destination is at 1000,500 and our mouse is at 0,0, then the delta is 1.0,0.5). The delta will always decrease to 0,0 as that is the pixel we're moving to. It's easier for the network to learn to decay to 0,0 from a starting delta, than to go from 0,0 to an end point. Now there's still a bit of jerkiness on the first position, most likely due to not enough data, so I also apply a small amount of smoothing to the points to reduce it.

    I'm using tensorflow (python) for the network.

     

    Gathering data:

    I made a simple program that records all mouse movements. When the square is clicked, it uses the destination as the end point, and then re-creates the nice delta data from there, then resets and moves the square to another random position. The reason why the square is so large is because it leads to more fluid and free mouse movements.

    image.png.4c58a74961e694aa89d61afe6783127a.png

     

    Plans:

    Once I get things smoothed out I'll try to implement it in some of my private scripts, and then open-source it. I'm also currently testing how it affects ban rates on brand new F2P bots, compared to a control set.

    Link to comment
    Share on other sites

    On 6/30/2019 at 12:37 PM, yeeter01 said:

    What is the performance hit on implementing something such as this tho?

    Pretty much 0

    Edit: Tested to be around 0.5-1ms per path generated.

    Link to comment
    Share on other sites

    • 4 months later...
    6 hours ago, Mimzy said:

    Hey, I have a few questions regarding this if you have a few minutes to spare. I sent you a friend request on discord (Edaredan).

    I just added you on discord.

    Link to comment
    Share on other sites

    • 2 weeks later...
    • 4 months later...
    • 1 month later...
    On 6/29/2019 at 10:31 PM, beezdul said:

    Video

    This is a small project I'm working on where I use a neural network to create mouse movements. In theory, the movements themselves should be completely undetectable (and the results are pretty impressive and look almost exactly like my real movements). Currently I've trained it on about 4000 clicks from my own data.

     

    How it works:

    The network is a 3 layer fully connected network. The input is 6 float values (random value 0.000-0.001,last start delta X,last start delta Y,current delta X,current delta Y,1.0)

    The network has a 1000 neuron layer connected to a 400 neuron layer (trained with 50% dropout), and the activation function is leaky relu (tf.nn.leaky_relu).

    The outputs are 456 values, each ranging from -1 to 1. The output is grouped into 152 sets of 3, being the delta x, delta y, and either 1 or -1 (-1 signals that the movement is complete).

    The deltas are an amount from the end position, divided by 1000. (So if the destination is at 1000,500 and our mouse is at 0,0, then the delta is 1.0,0.5). The delta will always decrease to 0,0 as that is the pixel we're moving to. It's easier for the network to learn to decay to 0,0 from a starting delta, than to go from 0,0 to an end point. Now there's still a bit of jerkiness on the first position, most likely due to not enough data, so I also apply a small amount of smoothing to the points to reduce it.

    I'm using tensorflow (python) for the network.

     

    Gathering data:

    I made a simple program that records all mouse movements. When the square is clicked, it uses the destination as the end point, and then re-creates the nice delta data from there, then resets and moves the square to another random position. The reason why the square is so large is because it leads to more fluid and free mouse movements.

    image.png.4c58a74961e694aa89d61afe6783127a.png

     

    Plans:

    Once I get things smoothed out I'll try to implement it in some of my private scripts, and then open-source it. I'm also currently testing how it affects ban rates on brand new F2P bots, compared to a control set.

    is this still happening? lmk if i can help or continue from where you stopped.

    Link to comment
    Share on other sites

    Archived

    This topic is now archived and is closed to further replies.

    ×
    ×
    • Create New...

    Important Information

    We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.