The biggest AI hardware vendors are splitting chips apart to build a new ecosystem around inference.