Galaxy clusters contain a large population of low mass dwarf elliptical galaxies whose exact origin is unclear: their colors, structural properties, kinematics and globular cluster population differ substantially from those of dwarf irregulars in the field. We use the Illustris cosmological simulation to study the different properties and assembly paths of dwarf galaxies according to their environment. We find that cluster dwarfs form earlier than those isolated in the field, naturally reproducing their redder colors. We argue that starbursts events associated to the environment together with the early assembly of cluster dwarfs can provide a natural explanation for the higher specific frequency of globular clusters in dwarfs in clusters, as found observationally. The origin of dwarf ellipticals in clusters is, therefore, consistent with an environmentally-driven evolution of field dwarf irregulars.