BOUNTY (30USD): Side-wide search function with keyword save-up 🤑

[justshipit]

Hey!

Another bounty from me (I will actually pay you for a solution). I am looking to integrate side-wide AJAX search on my site and want to have an option to save the search queries people type in and send them over to another google spreadsheets (or somewhere else I can easily view them).

My main bottleneck so far is that I pull my data from Google Spreadsheets (I use ‘middleman-spreadsheet’ gem for that), not like normally you would pull from data directory so generating the index and map will be a bit different from the example included on the Github (see link below).

So there are 2 parts of this project:
part a) use external Google spreadsheet as a database, allow the user to input the search query and find related posts **based on the generated JSON file with the snapshot of google spreadsheet data. I don’t mean the search function to work purely based on google spreadsheet. Google spreadsheet it’s just a database where I keep data but then we still need to have JSON snapshot that is generated by running middleman build

part b) create a function where each new search query will be submitted to another spreadsheet (spreadsheet different from the one with data under part 1).

Here is the demo of what exactly I want to achieve (so when you type, it automatically crawls through the content and find related items):
https://docs.shiftr.io/

Here is the gem they are using:

256dpi/middleman-lunr

Help!

[iwarner]

Hiya

The gem is easy to setup, and I save into google analytics

Can expose some sample code

I will try and write a blog post and u can donate the cash

Ian

[justshipit]

Hey Ian, I’ve updated my initial post a bit.

P.S. How’s progress? Did you have a chance to sit into it?

[sgnl]

I didn’t want to sleep, so I attempted it. Lunr doesn’t like the shape of my data, or it’s incorrectly generated.

To test, I’ve been calling the loadIndex and queryIndex("thing to search for") on the client-side console.

Almost there? Tag, someone else is it

spreadsheet: https://docs.google.com/spreadsheets/d/1laHzHMvlNdhK3Adf0W8c61OMW4zEv4EnMIpIOmoxQ7E/edit?usp=sharing

repo: https://github.com/sgnl/spreadsheet-middleman-json-search

[justshipit]

@sgnl

I didn’t want to sleep, so I attempted it. Lunr doesn’t like the shape of my data, or it’s incorrectly generated.

To test, I’ve been calling the loadIndex and queryIndex(“thing to search for”) on the client-side console.

Almost there? Tag, someone else is it

spreadsheet: https://docs.google.com/spreadsheets/d/1laHzHMvlNdhK3Adf0W8c61OMW4zEv4EnMIpIOmoxQ7E/edit?usp=sharing

repo: https://github.com/sgnl/spreadsheet-middleman-json-search

Hey sgnl!

Thanks so much for your attempt. I am going to check this out, if I can figure out the missing part and it all runs correctly, I will PayPal you the money.

[justshipit]

@sgnl

I didn’t want to sleep, so I attempted it. Lunr doesn’t like the shape of my data, or it’s incorrectly generated.

To test, I’ve been calling the loadIndex and queryIndex(“thing to search for”) on the client-side console.

Almost there? Tag, someone else is it

spreadsheet: https://docs.google.com/spreadsheets/d/1laHzHMvlNdhK3Adf0W8c61OMW4zEv4EnMIpIOmoxQ7E/edit?usp=sharing

repo: https://github.com/sgnl/spreadsheet-middleman-json-search

Hey! I ran this but I don’t see there any user interface where you can input the search query and crawl through the content. (like in the gif attached in my initial post). How can I access it?

I’ve edited my post to make the scope of work more clear.

[sgnl]

I did forget to include an example of the file I ignored as well. Thanks for the clarification. Got it, I’ll see if I can get it up to spec

[sgnl]

@justshipit

use external Google spreadsheet as a database, allow the user to input the search query and find related posts in live-thread

This is the part that bothers me. Client-side javascript will be needed to query for live data. Will need to do this with bare Lunar.js and AJAX to Google Spreadsheets. Same with:

create a function where each new search query will be submitted to another spreadsheet (spreadsheet different from the one with data under part 1).

Part B will need client side javascript as well since middleman is just a static file server (no back-end logic).

Otherwise, the solution I’m currently heading towards only works for static data. Since the JSON file will be created when middleman build is performed and it’ll be just a snapshot of the data at that moment of the spreadsheet on GDrive. I feel like this is totally not what you want lol.

IMO, your goals can be achieved with a client-side JS solution.

[justshipit]

@sgnl

Otherwise, the solution I’m currently heading towards only works for static data. Since the JSON file will be created when middleman build is performed and it’ll be just a snapshot of the data at that moment of the spreadsheet on GDrive. I feel like this is totally not what you want lol.

Actually, that’s what I want I understand the JSON is created by running middleman build command and I don’t expect the search function to be based purely on the content of the Google spreadsheet.

It’s absolutely fine if the search is based on that generated JSON file with the snapshot only.

The reason why it wouldn’t even make sense for me to be based on the spreadsheet alone is - that spreadsheet will include information about 1000s of separate pages (so URL slugs, titles, and content) that needs to be created so the website can have content actually.

@sgnl

Part B will need client side javascript as well since middleman is just a static file server (no back-end logic).

Yes, client JS is enough.

P.S. Appreciate your hard work so far.

[justshipit]

Bumped the bounty from 20 to 30USD. Keen to have the solution as my site is up and running already.

[justshipit]

(Shameless bump)

Any takers?

[justshipit]

Still no solution. Would love someone to pick it up in their free time!

[andreimoment]

Am I understanding correctly:

• Middleman generates content from a google spreadsheet
• You would like to add search based on that content
• Search could be lunr.js powered, based on a json file created at the time of the build

1. You looking for a way to generate the json file
2. You are looking for the front-end code integrating solr.js with your search, including CSS etc.?
3. Search is based on page title and description only? Or are you hoping to include the full body?

Are both 1 & 2 correct?

[justshipit]

@andreimoment

Am I understanding correctly:

Middleman generates content from a google spreadsheet
You would like to add search based on that content
The search could be lunr.js powered, based on a JSON file created at the time of the build

You looking for a way to generate the JSON file
You are looking for the front-end code integrating solr.js with your search, including CSS etc.?
Is search based on page title and description only? Or are you hoping to include the full body?

Are both 1 & 2 correct?

1 - I guess JSON file is already generated from the spreadsheet thanks to ‘middleman-spreadsheet’ gem as I can already build the whole website out of it. Just that, for some reasons, the lunr does not see this content correctly. And we need it so I can actually have the search function working.
2. I just need the barebone solution to it works like in the gif - I can style it later myself (CSS etc.)
3. Search is based on title, description and perhaps one more, an extra 3rd column from that spreadsheet (or rather - its snapshot). It could be tags column. No need to search based on the content of the article.

[andreimoment]

So, you need someone to

• troubleshoot the json file and its proper generation
• create the JS solution for the lunr search UI: update search on keyup, update list highlighting on up/down arrow, clear search on esc, go to first/highlighted search result on Enter or mouse click, do nothing if no results
• set up lunr.js to search an additional column, treating it as tags

This is an interesting problem. Unfortunately not enough for me personally to do for play/$30 bounty. If I build something like this for my own sites, I will be happy to share. Hopefully someone else has the free time to help.

[justshipit]

I am bumping the bounty to 40USD

[tomrutgers]

I’ve once built something like it because I wanted to know if it can be done, but the prototype was made with jQuery and the UI Autocomplete function which is admittedly not the most elegant solution.

[justshipit]

@"tomrutgers

I’ve once built something like it because I wanted to know if it can be done, but the prototype was made with jQuery and the UI Autocomplete function which is admittedly not the most elegant solution.

@tomrutgers perhaps you could be based on what @sgnl did so far?

@iwarner & @sgnl did you make any progress since?

@andreimoment did you come up with similar search for your own site at the end?

[iwarner]

Hi I edited miiddleman-search and update to latest Lunr etc

You can see the search i built here - just searches the blog posts titles and heading
https://foodscene.deliveroo.co.uk/

My branch:

iwarner/middleman-search