We study the vertex algebras associated with Heisenberg algebras $\widehat{\mathfrak{h}}$, constructed from vector spaces $\mathfrak{h}$ equipped with a non-degenerate skew-symmetric bilinear form, over fields of prime characteristic. These vertex algebras are isomorphic to Weyl vertex algebras. In contrast to the characteristic zero case, Weyl vertex algebras are no longer simple. We then focus on the study of simple quotient vertex algebras. We prove that for each such simple quotient, irreducible modules are unique up to isomorphism, and every module is completely reducible.