October 6th
Today I learned a loose connection between Fourier analysis and theory of Dirichlet series and convolutions. The correct context to abstract both of these ideas is in abstract Fourier analysis. Namely, take $G$ a group; I think the correct hypotheses are locally compact abelian topological group, but I don't care that much. Write $\widehat G=\op{Hom}(G,\CC^\times)$ for its characters. Then we say the Fourier transform of a function $f:G\to G$ is another function $\mathcal F(f):\widehat G\to G$ by\[\mathcal F(f)(\chi)=\int_Gf(x)\overline{\chi(x)}\,d\mu(x)\]for some measure $\mu$ on our $G.$
This formula looks scary, but it's quite familiar. In the context of Fourier analysis over $\RR,$ our characters $\RR\to\CC^\times$ are $\exp$ and friends (you can make other choice-like characters, but preserving the topological properties of $\CC$ forces). However, we can quite easily enumerate these characters by $x\mapsto e^{2\pi isx}$ for some scale factor $s\in\CC.$ So we can actually express $\mathcal F(f)$ by\[\mathcal F(f)(s)=\int_\RR f(x)e^{-2\pi isx}\,dx.\]And viola, the Fourier transform appears. Abstract theory beind Fourier analysis (which I don't understand) can give us typical properties we like; e.g., applying the Fourier transform twice gives back the original function. While we're here, we remark that we have a Fourier function convolution by $\mathcal F(f*g)=\mathcal F(f)\mathcal F(g).$ This will be helpful momentarily.
The nice observation, now, is to apply this to $(\NN,\cdot)$ (it's not a group, but it still roughly works) in order to obtain theory of Dirichlet series and convolutions. What are our characters $\NN\to\CC^\times$? We're going to cheat a bit and require these to also be characters over $(\RR^+,\cdot),$ presumably so that continuity and other things work out. (I don't entirely understand why this is the correct thing to do.) In particular, we can write, for character $\chi,$\[\chi(n)=\chi(e)^{\log n}=n^{\log\chi(e)}.\]In particular, we can parameterize all of our characters by $s\in\CC$: it's the one taking $n\mapsto\chi(n)=n^s.$ Then our Fourier transform looks like\[\mathcal F(f)(s)=\int_\NN f(n)n^{-s}\,dn.\]An integral over $n$ is really an infinite sum. Namely, we see\[\mathcal F(f)(s)=\sum_{n=1}^\infty\frac{f(n)}{n^s}.\]And viola, the Dirichlet series for $f$ appears. Of note is that $\mathcal F(\op{id})=\zeta$ and $\mathcal F(\delta)=1,$ where $\delta$ is the $1$-indicator.
Relating this to convolutions, we still write $f*g$ as our function satisfying $\mathcal F(f*g)=\mathcal F(f)\cdot\mathcal F(g).$ For example, this tells us that $\delta$ is the identity of this operation. And more generally, taking products,\[\sum_{n=1}^\infty\frac{(f*g)(n)}{n^s}=\mathcal F(f*g)=\mathcal F(f)\mathcal F(g)=\left(\sum_{n=1}^\infty\frac{f(n)}{n^s}\right)\left(\sum_{n=1}^\infty\frac{g(n)}{n^s}\right).\]In particular, expanding tells us that\[\sum_{n=1}^\infty\frac{(f*g)(n)}{n^s}=\sum_{n=1}^\infty\left(\sum_{ab=n}f(a)g(b)\right)\frac1{n^s}.\]So this motivates defining $(f*g)(n)=\sum_{ab=n}f(a)g(b),$ the definition of the Dirichlet convolution! And then once we see $\mu*\op{id}=\delta,$ we get M\"obius inversion, as usual.