Yes, you are right. After adding an external mailbox as one Office 365 account’s connected account, the emails will still in your external mailbox when you delete them in Office 365.
Based on my test, you can delete this connected account and then add it with the POP connection type. When you add this connected account, please type the wrong password. After that, you will get the following screenshot:
And then click POP, you will get the following screenshot, please note: uncheck the Leave mail on server check box.
After that, all the emails will be downloaded to your Office 365 account, the downloaded emails won’t leave in your external mailbox.
Based on my test, you can delete this connected account and then add it with the POP connection type. When you add this connected account, please type the wrong password. After that, you will get the following screenshot:
And then click POP, you will get the following screenshot, please note: uncheck the Leave mail on server check box.
After that, all the emails will be downloaded to your Office 365 account, the downloaded emails won’t leave in your external mailbox.