Yes, those are hard conversations to have. First, I want to applaud you both for choosing what is right for you both by traveling with your closest loved ones to have an incredible adventure. It takes courage, and you are doing what is right for your relationship and marriage! I'd encourage you to just be as intentional as possible so that they feel loved and honored, and they will understand For the people not invited, there are several ways to still include them! I'll list a few general ideas below.
1) Share your reasons why you are choosing a destination and can't bring them along.
Maybe you are saving money to buy a house, choosing to invest in your honeymoon, we don't want the stress of planning a big wedding and catering to many people (plus the cost of that) or other reasons. I understand that being close to your church family has made this probably a bit harder, but choose your closessssst people and still celebrate with others. I know this sounds silly, but it sort of reminds me of Jesus and his disciples. He had a whole flock of followers, and then 12 that were his people, but he chose 3 that he truly invested in. Jesus gave them the privilege of laying face down in the ground before the thunderous voice of God, as well as many other things! I like to think of describing smaller weddings like this - your church family is large and deep, but there are a few groups that go much deeper!
2) You can still have a "We Got Married!" party!
Perhaps you can have a mini reception back home in your backyard with chipotle and swimming? Something way more relaxed to celebrate! Likewise, you can still have any party with them! Joe and I had a reception for his small town. We used a family friend's property, bought some casual food from Sams, and did a meet-n-greet for 4 hours. I've heard of girls doing an epic bachelorette, or a coed Escape Room party, and others. I've heard of people having parties and it ended with watching their wedding video, or a slideshow of their wedding images.
3) Include them in your wedding day - virtually.
Your people can either be Facetimed in (assuming there is wifi) or have them record messages, write letters, open gifts - anything to still make them feel comfortable. If you have more time during your reception, you can easily do 30 minutes of Facetiming groups of people to thank them for all their love and support! Maybe you have a friend who is amazing at calligraphy or graphic design, and they can make a "just married" banner for your hiking backpack.
Also, a photographer I love said that she tells couples (who are planning a bigger party back home after their wedding) to make invitations that say, "We are going to say 'I Do' and then party with you!" The invite would give more details, but it still literally makes them feel invited.
And lastly, we all mean so much to each other as brothers and sisters in Christ, but you don't technically "owe" anyone an invitation to your wedding simply because of current relationships. I know that might read a little harsh, but I've seen a lot of people hurt by their friends, and oftentimes, those friends wanted to attend for selfish reasons and then pushed the couple's boundaries by saying they would do XYZ to attend. It is such a difficult conversation, especially since I don't know your friends, but it's as simple as asking yourself how important is it for you to have to plan, cater, pay, provide, converse, and celebrate alongside more people than you are really able to/capable of - whether that is emotionally, financially, mentally. People who truly love and support your decision to get married to the love of your life will understand why it is so important for you and will cheer you on! Lastly, a wedding is one day. A marriage is a lifetime, so invest in the things that will benefit your marriage and life together!