The man is devastated that Byuu didn't even say goodbye, and his parting note was to some random online weirdo who didn't even know Byuu was married, who listened to him announce he was suicidal and taking an overdose - and didn't alert the authorities until HOURS and HOURS after he died (and didn't know Byuu well enough to know his address).
Have you ever deal with intestacy? When someone dies without a will? It is a nightmare. Not only are you bereaved and having to go through probate, everything is more complicated because there's no will. Now try doing that in a foreign country in a different timezone with completely different bureaucracy that speaks an utterly incomprehensible language when you have no money.
He wanted Byuu cremated in Japan and scattered somewhere beautiful that Byuu liked. That's not because he didn't give a fuck. It's because he was heartbroken that the love of his life was a selfish venial man who didn't give a shit about him. It reads to me that he wanted to selflessly try to honour Byuu's last wishes - Byuu obviously had no desire to return to Ohio, no links back to the home country with his estranged family all dead, and the only seemingly meaningful connections - his husband and this sobodash person - viewed as irrelevant. So let him stay in a scenic grove overlooking the sparkling waters of Tokyo Bay, maybe somewhere Byuu went to get away from all the hustle and bustle because he obviously didn't do well around crowds. Except the correspondent basically says in those documents that Byuu never travelled and so didn't have anywhere "he liked to go", he just sat in his room talking to internet strangers who knew nothing about him while eating illegal estrogen tablets.
Naturally the husband also wanted access to the Japanese accounts and details of insurance policies etc because he'd need money (that he didn't have) to even consider recovering any possessions.