If you remove the shiny surface of the rubber, then using a strong CA glue should hold the wood in place. I just tested it out using Loctite 401.
Not sure if you have it over there but heres a link to a site over here about Loctite 401 :
LinkLoctite 401 = best glue ever.