Kaye starred at Glendora High School in his hometown of Glendora, Calif., hitting .331 over the course of his four-year career with the Tartans. That production drew the attention of professional and collegiate talent evaluators, eventually leading the Texas Rangers to select Kaye in the 15th round of the 2015 MLB Draft.

After passing on his commitment to play collegiately at Cal State Fullerton and signing with the Rangers, the left-handed hitter was assigned to the Rangers Arizona League affiliate for his professional debut in 2015. Kaye recorded 18 hits in 36 games in his rookie campaign, then repeated the level in 2016, raising his average more than 75 points. Over the course of 40 games, Kaye hit .250 with six doubles, two triples, and 21 RBI, adding 22 walks for a healthy .369 on-base percentage. That performance earned Kaye a promotion to Low Class-A Spokane for the 2017 season. The 21-year old notched nine hits over 34 games with the Indians, including two doubles and five RBI. While primarily a corner outfielder, Kaye also saw brief stints in centerfield and at first base with Spokane.