Yes. Go to Settings → Apps → Play Store → Uninstall updates. But downgrading is not recommended as older versions break app updates.
Make sure you download the correct armeabi-v7a or universal version for Android 5.1, and enable "Unknown sources" before installing. Also, this is just the Store app – if your core Play Services are too old, some features might not work. some features might not work.