Why your own Facebook group scraper stops working

Writing it is the cheap part

The first version comes together quickly and works. That is why the decision to build it almost always looks sound in the week it is taken. The cost that matters arrives later and spreads over months, so nobody sets it against an invoice they never paid.

Count it differently: not what it costs to write, but what one day of it not working costs, multiplied by how many such days a year there are.

The failure is silent

When something like this breaks, usually nothing crashes. There is no red alert and no log line anybody looks at. Enquiries simply stop arriving — and no notification looks exactly like a quiet week in the groups.

So these failures are found late and usually by accident: somebody notices a month on that one channel produced nothing. By then the enquiries posted in the meantime belong to somebody else.

The changes come on somebody else's calendar

Facebook publishes no stable interface for this content and announces no changes to it. Which means every rebuild on their side is unplanned work on yours — arriving when it arrives, not when you have room for it.

In practice that is the whole difference: buying a tool buys somebody else's Friday evening. Building it reserves one of yours, and you do not yet know which.

Who keeps it running when its author leaves

Projects like this are usually written by one person, after hours, with enthusiasm. Six months later the sales pipeline depends on something exactly one person understands, with no documentation and no second owner.

That is the risk no budget shows, because there is no line for it. It materialises in the week that person changes jobs — and then the cost is not developer time, it is a pipeline that has stopped filling.

When building it yourself is the right call

There are situations where this is a good decision, and it is only fair to name them. If you have an experienced person with a genuinely free quarter, if monitoring content is part of what you sell to your own clients, and if you can point at the second person who will take the code over when the first one leaves — build it. Then the maintenance is not a side cost, it is investment in a capability you want in the building anyway.

The opposite case is just as easy to recognise. If this would serve only your own sales, if it would be written after hours alongside other duties, and if nobody would notice for a week that it had stopped — that is not a project, it is a debt being taken on. In that arrangement the difference between building and buying is not about money; it is about who carries the risk of a silent gap in enquiries.

Set an output baseline

Do not monitor only whether a scheduled run finished. Monitor whether it returned a plausible amount of current content. Pick five public groups or pages that normally produce posts. For each source, record the usual result count, newest visible post date, and one sample matched phrase. Check those fields every working day. A completed run with no current posts is a result to investigate, not a success. Keep the baseline in a shared sheet so the check survives one person’s absence. Change it only after a documented source change.

Use a simple alert rule

Write alert rules before you need them. Flag a source when it returns nothing on two consecutive checks, when its newest result is older than its normal pattern, or when several sources drop together. Send the alert to a shared inbox or team channel, not one developer’s address. Include the source link, last result date, previous result count, and check time. Keep alerts separate from new-lead notifications. Otherwise a busy day of leads can hide the warning that collection has stopped. Review false alerts monthly and adjust the baseline.

Practice the recovery

Create a short recovery checklist and test it on one public source. First, open the source signed out and confirm it is still public. Next, compare the newest visible posts with the stored output. Check whether the reader missed all posts, missed comments, or returned stale results. Save a dated example showing the mismatch. Pause lead notifications for that source if they would send misleading results. Assign one person to investigate and another to verify the fix against the saved example. Only resume the source after the newest expected content appears again.