Syncing contacts over usb cable

You have a linux machicne so you could host your own private cloud cloud: see the ecloud-selfhosting repo .
Some posts in the forums too: this one looks useful.
Good luck!

[Added later] Doesn’t have to be eCloud. You could run NextCoud server instead: the /e/ software supports conacts and calendar sync with NextCloud out of the box. And lots of information around about running NextCloud server on Debian, e.g. here or here