So if you were to answer simply the key concerns from TLJ, what would you say to:
1. Why couldn't the rebels in ROTJ have simply sent the Endor shield team down to deactivate it like they do, but instead of endangering everyone in the rebel fleet, send say five of the largest (ie size of raddus or bigger) EMPTY spacecraft they could steal/find, piloted by a droid or auto pilot, into hyperspace and don't stop as the rebel fleet does in ROTJ, and plow right into/through the DSII?
As you've accounted for, having to covertly take the shields down on Endor first would be the key part of
any plan. Once the shield generator had been taken out, sending smaller fighters into a WIP Death Star (thereby providing easy access) was a solid strategy with a good risk/reward ratio. Having smaller fighter ships navigating through the internal DSII structure and taking aim at the core directly allows for far more precision and reliability than using a hyperspace kamikaze tactic. The plan in the movie would've been preferable from a logistical point of view. The Emperor being prepared for their arrival (which the Alliance couldn't have known) is the only reason that the operation got messy in ROTJ.
Holdo destroying the Supremacy wasn't an offensive strategic tactic to take out an enemy asset, it was a defensive tactic to preserve the remainder of the fleeing Resistance fighters. It was desperation. As a tactical option for attack, the risk/reward ratio (given the number of factors that need to be accounted for) would make it virtually a non-option in any other scenario. Some of those factors (pre-calculated hyperspace jump coordinates Poe had set, Holdo manually overriding safety protocols, the Raddus using experimental shields, etc.) that have been explicitly outlined in canon to contextualize how unlikely a similar tactic would be in succeeding under other circumstances has been well established. It worked (against heavy odds), but there was no other option on the table. In ROTJ, the Rebellion had come up with a tactical plan that I see as entirely superior to sacrificing key resources on a high-odds gamble.
We can't underestimate the massive size of the DSII. Even with an enormous ship like the Raddus, the structure of Snoke's Supremacy (which had a much smaller profile than a Death Star) was still merely split/severed. I don't see how the Rebellion leadership would've deemed a similar hyperspace kamikaze maneuver as being a better (or even worthwhile) option. And I agree with e30ernest about how the Rebellion would be hesitant to waste their best resources. They were the galaxy's only chance at toppling the Empire; preserving their fleet (especially their costliest assets) would've been a priority. Destroying the first Death Star didn't cripple the Empire; far from it - they just immediately started building a second one. The DSII needed to be destroyed, but doing so with a Holdo tactic would've only been an option as a an act of desperation if the main plan failed.
2. Why didn't anyone use force projection in the OT, even for a short "non-fatal" trip (say a minute or so,) for example Ben force-projecting to Alderaan for a minute to tell the rebels about leia's capture and the DSI plans, or Yoda in ESB force-projecting to Bespin to aid the about-to-die Luke (even just enough for Luke to escape ), or Vader using force projection at many points in the OT?
Since you've made it clear that on-screen OT is the only reference source that you consider the biblical-type authority in establishing the rules of SW, I'll use on-screen OT context exclusively in my answer for this one.
And that context has a lot to do with how the idea of "Force ghosts" gets portrayed in the OT.
To me, it was ANH (not the PT) that established Force ghosts as an actual Force ability (rather than as our universally-understood concept of "conventional" ghosts/spirits). When Kenobi gave himself up at the end of the duel with Vader, the earlier line of
"I shall become more powerful than you can possibly imagine" is more telling than it seems on the surface. Remember, Obi-Wan is talking to a fellow Force user with a "devotion to that ancient religion." Why wouldn't Vader (a fellow master of the Force) be able to "possibly imagine" the potential of a Force ghost? It was because Obi-Wan clearly felt that he knew something that his former pupil didn't/couldn't understand.
When Kenobi projected his voice to encourage Luke to "run Luke, run," he was immediately demonstrating why he had a knowing smile before his body vanished. By surmising that he'd become something unimaginably powerful, Obi-Wan was establishing that what he was about to achieve had not been seen/known by even the likes of a master of the Force (surely, Vader/Anakin would've been aware of Force ghosts if all Jedi had transitioned that way in the past). Kenobi was breaking new ground; he was mastering a previously-unseen Force ability. And yes, in the PT we discover that he had specifically learned it from Qui-Gon. A Jedi Master had discovered and passed on a new ability.
In TLJ, Kylo is the only one on screen (outside of Luke) to demonstrate any awareness of true Force projection and its lethal consequences. Since Kylo/Ben had been trained by Luke, we have yet another case of a master training his apprentice about a new/unused Force ability. Even more fitting/poetic is the fact that Luke has become the Obi-Wan of the ST.
The way I interpret the projection in TLJ, Luke surrendered himself to the Force in order to accomplish the feat. In essence, Luke had to surrender his corporeal existence (much like Obi-Wan did in ANH) in order to project himself at all in another form (again, like Obi-Wan). When Luke dissolved the projection, I view those last moments of his on Ahch-To as him clinging on as long as possible in order to properly say goodbye to not only Leia and Rey, but to his physical life. For all intents and purposes, corporeal Luke was effectively gone as soon as he transitioned into projection form. His full physical departure in front of the binary suns was a delayed closure for Luke (and for the audience).
Kenobi, Yoda, and Vader never used Force projection because 1.) Luke and Kylo are the only two who we can be sure even knew about it, and 2.) it would mean having to surrender yourself to the Force; aka dying in the physical world being the inevitable consequence.