Dale you clever guy. I'll bet your idea can work -- not only to give you a relative measure of fabric permeability, but you could probably approximate lab measurements of cfm as well. As long as your measurements are repeatable, you can probably covert to industry standards without spending for a fancy machine. More on that in a bit.
Your source of air -- vacuum on "blow", air compressor turned to a specific psi, strong fan, whatever -- just make sure it's repeatable.
Calibrate. Start with an average wind shirt (what's average? 5cfm? 10cfm?). Increase/decrease distance of air source and/or inspirometer until you're getting measurements in the middle of the range on your inspirometer -- that would be around 2000.
Test several (a dozen?) jackets with *known* permeability in a common measurement (take what you can get: cfm/grams per meter/Ret/whatever). Let's assume cfm for now. The jackets have numbers of 5,10,4,20 cfm, etc... Record you inspirometer readings for same. Just spitballing some numbers, they might be 200, 3000,1500, 4000, etc, or whatever.
Once you have these two columns of numbers for a dozen jackets, you can predict the published cfm numbers (DVs) using your inspirometer numbers as independent variables (IVs) with a statistical regression. The regression will even give you a "fit" number telling you how well you are predicting the published numbers. If you are getting an r-squared of .90 or more, you have a darned accurate home testing machine. I'd be happy with an r-squared of .80 for a project like this, where gilt-edged accuracy is not the goal.
The regression will give you a simple formula (often just two variables, one to add and one to multiply), where you can take your inspirometer numbers, plug them into the regression formula, and it will yield estimates of cfm (or whatever metric you chose). Thus, a simple calculation will allow you to estimate permeability of new fabrics you encounter.
I did something similar once when estimating lumens for flashlights. I don't own a fancy laboratory integrating sphere, but I cobbled up some PVC pipe, using it with my existing light meter, which allowed consistent measurements of indirect light in a confined space. After testing maybe 18 lights with known/published lumen amounts, I was able to predict lights with and r-squared above .9, meaning that with some small amount of "slop" in the system (10%), I was getting impressively accurate lumen estimations without more than $3 invested for the PVC pipe.