Depends where you are and how much proof they have. But generally speaking, yes it is a criminal offense and can result in arrest and prosecution. It's considered to be a form of harassment, intimidation, or animal cruelty.
A general rule of thumb to live by is to never voice your true intention ever. Keep them guessing. So when you truly decide to kill off that annoying dog, no one has proof of you having motive or anything related to that.
If you truly want to kill a dog specifically and get away with it, there's a simple way to do so that's quick, easy, and efficient. Don't bother killing it via blunt force trauma or using a knife to stab it to death. Instead utilize Xylitol, which is a sugar substitute commonly used in sugar-free gum, candy, and baked goods, which is something that is especially dangerous for dogs. Even a small amount of it can cause death in dogs. This is because when a dog ingests xylitol, it can cause a rapid insulin release, which can lead to hypoglycemia (low blood sugar). This can result in seizures, coma, and even death if not treated immediately. Additionally, xylitol can cause liver damage in dogs, which can also be life-threatening.
Now, about how you would get the dog to ingest the Xylitol: you could use it in conjunction with other foods that attract dogs. A possible food you could use are anise seeds. From my experience, dogs are very commonly attracted to them. That's a simple way to lure dogs to ingest the Xylitol, hence killing them or heavily maiming them.