It's not magic - it's merely an extension of the Sorian AI method. LOUD adds additional category definitions to those that were supported by Sorian so that it can better understand how units are to be used. This is of particular note for amphibious units which were often built as part of the standard unit choices (particularily in T3 where the original definitions had no amphibious choice at all). In those cases where a custom unit is categorized along with existing units, the custom unit file can specify a percentage chance that the AI will select that unit rather than an existing one.
For experimentals, this idea has been greatly expanded. There are now 4 levels for Land, 2 for Air and 2 for Naval. Artillery and Economic Experimentals are handled independently. The levels are primarily determined by cost, so that the AI can choose one that suits his current eco.