Many common finite p-groups admit non-trivial automorphisms of order coprime to p, However when p is odd, it is reasonably difficult to find finite p-groups with automorphism group a p-group. I'll talk about joint work with Ursula Martin proving that almost all finite p-groups do have automorphism group a p-group when p is odd. The asymptotic sense in which the theorem holds involves bounding the Frattini length of the p-groups and letting the number of generators go to infinity. The proof of the theorem draws on a detailed analysis of the Frattini series of a free group and the combinatorics linking finite p-groups and representations of GL(n,p). The case of p = 2 remains open.