Google’s John Mueller said on Twitter that Google Search does not hold a grudge for SEO technical issues you have on your site. When you fix those issues and Google reprocesses the affected pages or site, things should return back to normal.