Except he also says you were the only courier who used to go to the divide, that through your delivering of messages you kept the divide connected to the rest of the world and alive, that you went there more than once when no other courier would thus making the divide your town, possibly your home. And then he says you delivered a package that caused the big badaboom.

I saw no dialog line that let me flat out deny it, only a few that said that maybe i didn't remember it too well. Had he said that the courier went to the divide only *once* it could make sense that the courier wouldn't remember that location or package in particular.
But if he also says that you're the one who used to go to the divide when all the other couriers refused then the only non-amnesiac explanation for us not knowing anything about the divide is that ulysses has the wrong guy. Which isn't far fetched considering he comes across as burned out druggie(unintentionally?), however at no point do we get a line to flat out deny that it was us.

What we do get is a shitload of dialog lines where we say "maybe, wouldn't remember" and also a ton of dialog lines where we express our ignorance about the divide or our trips there or the delivery.