Random Tree Feature Layer

1. Integration point: add a non-learned RandomTreeFeatures module immediately before the prediction MLP. Normalize every continuous feature to [0,1]; represent categorical variables by integer IDs or one-hot vectors. The module should output all leaf indicators for a ridge baseline and optionally a compressed vector for an MLP. 2. Pseudocode: sample T trees; for each tree t sample depth D_t, feature indices V[t,k], and cutpoints S[t,k]. For each minibatch x, evaluate every split b[t,k]=(x[:,V[t,k]]>S[t,k]), convert the D branch bits into a leaf index, and emit a one-hot vector psi_t with one at that leaf. Concatenate the vectors over trees and multiply by 1/sqrt(T); feed concat(x,z) to a two- or three-layer MLP. Alternatively solve ridge regression directly on the encoded training matrix. 3. Computed from the mathematics: partition indicators, normalization, and empirical kernel K_T^(T). Estimated empirically: depth, feature and cutpoint distributions, T, and ridge coefficient lambda. Compare kernels at T=128, 512, and 2048 against a large-T reference. For memory control, retain only reached-leaf IDs, use sparse tensors, or hash leaf indices into a fixed-dimensional embedding. 4. First experiment: use California Housing or UCI Adult with fixed train, validation, and test splits. Compare a raw two-layer MLP, LightGBM/XGBoost, and the augmented MLP with T=256 random trees at depths 4, 6, and 8. Measure test RMSE or accuracy, wall-clock training time, feature-construction time, parameter count, and performance at reduced training-set sizes. A positive result is lower test error than the raw MLP at equal optimization steps, particularly in the 1k-10k example regime, with tree-feature construction costing less than one boosting pass. Also test whether accuracy and kernel estimates stabilize smoothly as T increases.