Drivers are not a Linux thing. [1]
Did you use my suggestion to satisfy yourself that the lack of Driver created 
Problem device and that it really was a Drivers problem ?
Maybe there is a bug in /e/OS installer, idk, but there are several reports like this, /e/OS installer is new and I have not seen this happen myself. /e/OS installer seems not to give a good clue what is wrong; I would be guessing if I made suggestions.
Please feel free to send images or any further clues.
[1] Elsewhere I made this comment
If you follow that thread, you will see the marked solution where I recommended the manual method. fp3 is installed by a script so is not so hard Install /e/OS on a Fairphone FP3/3+ - “FP3”.
There is also this guide [HOW-TO] Flash /e/-OS on Fairphone 3 using Debian based GNU/Linux, quite old but updated.