FREE NEWSLETTER
TWO THINGS HEAVILY influenced my financial life. The first was my short stint after college as an internal revenue agent with the IRS. The second was getting married and having five children.
Result: I’ve spent most of my adult life as a tax-averse junky using retirement accounts to get my high, so much so that there’s a risk our retirement-account withdrawals will put us in a much higher tax bracket than when we made our contributions. I’m age 67 and Cindy is 58, and today we have 98% of our financial assets in retirement accounts.
I was an internal revenue agent from 1981 to 1983. I quit because it seemed everybody who I met either hated me or was scared to death of me. Still, the job convinced me that I should legally avoid taxes at all costs. But how? Starting in 1981, 401(k)s became fairly common. The following year, traditional IRAs became available to workers with pensions. That’s when I started funding IRAs, which were then limited to $2,000 a year. I really liked the idea of avoiding taxes until retirement, which at the time was some 40 years away.
I married at age 31 and my wife was 22. Opposites attract: I was a saver and Cindy was a spender. We often clashed over spending. We never made a lot of money and budgeting never worked for us. But we agreed on one important subject, and that was having children.
I noticed that whatever we put into our checking account would mysteriously disappear by the end of the month. There were a lot of arguments over the years. We both thought that our marriage wouldn’t make it. After all, money arguments are the No. 1 reason marriages fail.
After one bitter argument, I went into my payroll account and increased the withholding for my 401(k) by a couple of percent. It made me feel better, and it seemed preferable to arguing over how much we spent. I later did the same with my wife’s withholding.
After we started having kids, it didn’t make sense for Cindy to continue working full-time. She worked one day a week as a bank teller for 12 years, before going back full-time. While she was working part-time, I made sure that we contributed the maximum each year to both our IRAs. I also contributed a large percentage to my 401(k).
It was at this time that I started buying tax software. Before we would file our taxes, I’d use the software to run “what if” scenarios, so we could use our child tax credits in conjunction with retirement-account contributions to zero out our tax liability.
Eventually, because of our large tax-deductible retirement contributions, we got to the point where we couldn’t use all of our tax credits. Still, we continued to make those retirement contributions. But we also started converting traditional IRAs to Roths, which led to more potentially taxable income and allowed us to use up the credits without generating a tax liability. During this time, I was working up to three jobs, including my part-time employment in the Army Reserve, where I served until age 60. I retired from my civilian job at 59.
How did my wife take all this?
Cindy didn’t really like it, but somehow money put into retirement accounts would never get touched. Contributions served two purposes. They increased our retirement savings, while also limiting our spending. My feeling was, if things got too tight, we could always lower our retirement contributions or take a loan against our 401(k) balances. The loans only happened a few times for small amounts.
Our arguments over money grew less frequent. Any time I thought Cindy was spending too much, I’d remind myself that we were still super-saving for retirement, and my anger would pass.
Our five children picked up on our tight-money ways. They attended public colleges in Kentucky that are relatively inexpensive. They had part-time jobs and paid internships during high school and college. They also received multiple merit-based scholarships, though no non-merit aid. One lived at home during college.
Even though we gave our children little financial help toward college, each graduated with money in the bank and no debt. Our last child graduated when I was age 65. Four are engineers and one is an elementary school teacher. Three have master’s degrees. Believe me, we’ve had other major problems with our children, but handling money hasn’t been one of them.
We’ve continued converting our traditional IRAs to Roths long after our eligibility for the child tax credit passed. Yep, we’re actually paying taxes now, but keeping it in the 12% bracket. Today, about 21% of our retirement accounts are Roths. I plan on continuing to convert until I start drawing Social Security at age 70 or perhaps until required minimum distributions begin at 73—provided, that is, we can remain within the 12% tax bracket.
Cindy still works full-time. She contributes the max to her 401(k), and we also max out our IRAs. She’ll retire next July at age 59. We pretty much live on my Army Reserve pension, a tiny pension from a civilian job, and whatever is left from Cindy’s salary after her 401(k) contributions.
More disciplined people could no doubt have done better. But we’re both happy with how things have worked out. We now splurge more on our adult children and grandchildren. We can also eat out and travel without worrying about money. In fact, I suspect we enjoy these things more than most—because we’ve done so little of them in the past.
Ken Begley has worked for the IRS and as an accountant, a college director of student financial aid and a newspaper columnist, and he also spent 42 years on active and reserve service with the U.S. Navy and Army. Now retired, Ken likes to spend his time with his family, especially his grandchildren, and as a volunteer with Kentucky’s Marion County Veterans Honor Guard performing last rites at military funerals. Check out Ken’s earlier articles.
Want to receive our weekly newsletter? Sign up now. How about our daily alert about the site's latest posts? Join the list.
Thank you Ken. You were wise to increase contributions to retirement plans rather than argue with your wife. Like you, I am a saver and my wife is a spender. Yes, we argued too much about money when we had a joint checking account. Upon my decision, I created my own account so she could have hers without me knowing what she spent or why. Since she earned as much as I did career wise, this solution worked much, much better. As far as children go, our daughter takes more after me and our son more after my wife regarding saving money and investing in different retirement accounts.
Good article Ken. Decades ago I remember thinking that no way could our tax rate be higher in retirement, and happily increased our 401k contributions. As you can imagine a large portion of our wealth is in tax-deferred accounts. We’ve done some Roth conversions, but I will continue to look at them each year.
Thanks for your service to our country both in the military and as a revenue agent from a fellow Army veteran.
In the time of my active career in public accounting I represented clients before the IRS in only a handful of both office and field audits. I have now retired from such duties. Horror stories abound about abusive revenue agents in the popular press but that was almost never my experience in face to face audits. In the rare instance when an agent took what I consider an unreasonable position the IRS agent’s supervisor was then involved in the dispute and if such matter was not then resolved the IRS internal appeals process generally works if the taxpayer’s position is correct. While the formal legal process is also available the practical answer is the issue has to involve a lot of dollars or be a matter that the taxpayer or the IRS is unwilling to compromise on regardless of the legal costs. The only unwilling to compromise taxpayer position I was ever involved with dealt with involved the claiming a dependent by both parents after a bitter divorce. A good time was had by none.
Correspondence audit inquiries from the IRS are usually a result of mismatches between third party reports (think 1099s) the IRS receives and the reporting on the individual return in a place the IRS was not expecting or a taxpayer made an error or a inadvertent omission. The best and a majority of the audits I was involved resulted in a no change to the return as filed or were (relativity) quickly resolved with the taxpayer writing a check.
Only once in my career did I have one of a pair of agents show up for a face to face audit did one agent self identify as being from the criminal investigation division. At that point in a tax audit it is most likely time for the accountant taxpayer representative to end their active representation before the IRS and for the taxpayer to hire someone with a JD behind their name.
Best, Bill
Army 72-74, no active reserve duty
Bill, your mention of an uncompromising taxpayer after a bitter divorce was timely. On Tuesday I did a return for a divorced mother of 5. She and her ex had a verbal agreement on who would take which children as dependents, with an agreed upon plan as they children aged out. They are down to the last one, and she lamented that her husband had already claimed her. She also said her husband violated the agreement in previous years. The was unwilling to challenge him and directed me to fins her return with no dependents. But she was quite bitter.
I don’t know about the current college student aid program, but when our children attended college in the early 2000s our expected family contribution was rather low. How did we achieve this? Purely by accident. We were fairly good savers, but 90 plus percent of our savings were in retirement accounts. At least at that time none of your retirement account assets were used in calculating the family contribution towards college expenses.
Retirement accounts are still exempt from consideration for the parents’ contribution as per my recent review of my grandson’s FAFSA. Also, even custodial grandparents are not expected to contribute, unless they have legally adopted their grandchild.